Vision-2-vision control system

ABSTRACT

A method for controlling an object space having an associated object environment includes the steps of, defining a target set of coordinates in the object space, recognizing the presence of a predetermined object in the object space, and determining a coordinate location of the recognized predetermined object in the object space. The method further includes determining the spatial relationship between the recognized predetermined object and the target set of coordinates, comparing the spatial relationship with predetermined spatial relationship criteria, and if the determined spatial relationship criteria falls within the predetermined spatial relationship criteria, modifying the object space environment.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application claims priority of U.S. Provisional Application No.61/324,443, filed Apr. 15, 2010, and entitled VISION-2-VISION CONTROLSYSTEM, the specification of which is incorporated herein.

TECHNICAL FIELD

Embodiments of the invention are related in general to controllingoutput devices within an entertainment environment, and moreparticularly to controlling output devices based upon the detection ofobjects in the entertainment environment.

BACKGROUND

Modern stage lighting is a flexible tool in the production of theatre,dance, opera and other performance arts. Several different types ofstage lighting instruments are used in the pursuit of the variousprinciples or goals of lighting. Stage lighting has grown considerablyin recent years partially due to improved technical equipment. Lightingcontrol tools allow a user to change the quality of the lighting.Historically, this has been done by the use of intensity control.Technological advancements have made intensity control relativelysimple—solid state dimmers are controlled by one or more lightingcontrollers. Controllers are commonly lighting consoles designed forsophisticated control over very large numbers of dimmers or luminaires,but may be simpler devices which play back stored sequences of lightingstates with minimal user interfaces. Consoles are also referred to aslighting desks or light-boards. For larger shows or installations,multiple consoles are often used together and in some cases lightingcontrollers are combined or coordinated with controllers for sound,automated scenery, pyrotechnics and other effects to provide totalautomation of the entire show. DMX512 is the control protocol mostprevalent in the industry. Newer control protocols include RDM (remotedevice management) which adds management and status feedbackcapabilities to devices which use it while maintaining compatibilitywith DMX512, ArtNet which is an implementation of DMX over Ethernet, andArchitecture for Control Networks (ACN) which is a fully featuredmultiple controller networking protocol. These allow the possibility offeedback of position, state or fault conditions from units, whilstallowing much more detailed control of them.

SUMMARY

A method for controlling an object space having an associated objectenvironment includes the steps of, defining a target set of coordinatesin the object space, recognizing the presence of a predetermined objectin the object space, and determining a coordinate location of therecognized predetermined object in the object space. The method furtherincludes determining the spatial relationship between the recognizedpredetermined object and the target set of coordinates, comparing thespatial relationship with predetermined spatial relationship criteria,and if the determined spatial relationship criteria falls within thepredetermined spatial relationship criteria, modifying the object spaceenvironment.

A system for controlling an object space having an associated objectenvironment includes at least one processor configured to receive adefinition of a target set of coordinates in the object space, recognizethe presence of a predetermined object in the object space, anddetermine a coordinate location of the recognized predetermined objectin the object space. The at least one processor is further configured todetermine the spatial relationship between the recognized predeterminedobject and the target set of coordinates, compare the spatialrelationship with predetermined spatial relationship criteria, and ifthe determined spatial relationship criteria falls within thepredetermined spatial relationship criteria, modify the object spaceenvironment.

BRIEF DESCRIPTION OF THE DRAWINGS

For a more complete understanding, reference is now made to thefollowing description taken in conjunction with the accompanyingDrawings in which:

FIG. 1 illustrates an embodiment of a system block diagram for a controlsystem;

FIG. 2 illustrates an embodiment of an entertainment environment 200 foruse with the control system;

FIG. 3 illustrates an embodiment of general software components of thesoftware architecture for the VISION-2-VISION control system;

FIG. 4 illustrates an embodiment of a general overview of softwarelogical planes for the control system;

FIGS. 5-5′ illustrate an embodiment of logical elements and sequencesthat represent key control services capability of the control system;

FIG. 6 illustrates a particular embodiment of software components of theV2V-TRAX core services;

FIG. 7 illustrates a particular embodiment of software components of theV2V-IMAGE core services;

FIG. 8 illustrates a particular embodiment of software components forthe V2V-VOX core services;

FIG. 9 illustrates a particular embodiment of software components forthe V2V-SYNC core services;

FIG. 10 illustrates an embodiment of an example of a 4×4 LED panelmatrix under control of the V2V-SYNC software components;

FIG. 11 illustrates an embodiment of a user interface of the controlsystem;

FIG. 12 a illustrates a particular example embodiment of an operatorusing a laser pointer to designate a particular region of interest;

FIG. 12 b illustrates an embodiment of a user interface system with atouch screen where a user selects an object or object region of interestby touching the area on the screen interface;

FIGS. 13 a-13 a″ and 13 b-13 b″ illustrate an example embodiment of anoperator using a focus behavior function of the V2V-TRAX softwarecomponent, FIGS. 13 a-13 a″ show an example sequence of setting fixturefocus using a laser pointer, if the system is setup for focusvalidation, the system can validate focus attribute data using a systempositioning laser, as represented in example FIGS. 13 b-13 b″;

FIGS. 13 c-13 c″ show an example sequence for automatically settingfixture focus utilizing a system positioning laser;

FIGS. 13 d-13 d″ illustrate an embodiment of a flow diagram for focusbehavior;

FIGS. 14 a-14 a″ illustrate an example sequence for a gesture behaviorfunction of the V2V-TRAX software module: environment attributes areupdated based upon the observed objects gesture of pointing (otherpossible gestures include clap, jump, sit, stand, lay down);

FIGS. 14 b-14 b′ illustrate a flow chart of an embodiment of the gesturecontrol behavior;

FIGS. 15 a-15 a″ illustrate an example sequence of a proximity behaviorfunction of the V2V-TRAX software module where an environment lightfixture groups' attributes are controlled by the relationship between anobject and predefined location;

FIGS. 15 b-15 b″ illustrate an example sequence of the proximitybehavior function of the V2V-TRAX software module where an environmentLED panel is updated based the relationship of an observed object and apredefined zone or area;

FIGS. 15 c-15 c′″ illustrate an example sequence of the proximitybehavior function of the V2V-TRAX software module where a light, locatedin a predefined target zone, has its attributes updated based theattributes of an observed device (light) located in a predefinedreference zone;

FIGS. 15 d-15 d″ illustrate an example sequence of the proximitybehavior function of the V2V-TRAX software module where a discreteenvironment control device (e.g. alarm) attribute is updated based uponan observed object's relationship to a predefined trip line location;

FIGS. 15 e-15 e′ illustrate an embodiment of a flow chart for theproximity behavior function of the V2V-TRAX;

FIGS. 16 a-16 a″ illustrate an embodiment of tracking behavior functionfor the V2V-TRAX software module where lights in the environment areupdated based upon the observation of an object and its associatedposition;

FIGS. 16 b-16 b″ illustrate an embodiment of tracking behavior functionfor the V2V-TRAX software module where lights are being updated relativeto the position of an object in the environment where object selectionwas done by circling the object with a laser;

FIGS. 16 c-16 c′ illustrate an embodiment of a flow chart for thetracking behavior function of the V2V-TRAX;

FIGS. 17 a-17 a″, 17 b-17 b′, 17 c-17 c″, 17 d-17 d″, 17 e-17 e″, and 17f-17 f′ illustrate embodiments of a compare behavior function of aV2V-IMAGE software component;

FIGS. 18 a-18 a′ 18 b-18 b′, 18 c-18 c′, and 18 d illustrate embodimentsof command behavior function for the V2V-VOX software module;

FIG. 19 illustrates an embodiment of the audio behavior function of theV2V-SYNC software module;

FIGS. 20 a-20 a″, 20 b-20 b″, and 20 c-20 c′ illustrate embodiments of a2D_INIT mode behavior function of the V2V-TRAX software module; and

FIG. 21 illustrates an embodiment of a procedure for a profile behaviorfunction of the V2V-NOTE software component.

DETAILED DESCRIPTION

Referring now to the drawings, wherein like reference numbers are usedherein to designate like elements throughout, the various views andembodiments of a Vision-2-Vision (V2V) control system are illustratedand described, and other possible embodiments are described. The figuresare not necessarily drawn to scale, and in some instances the drawingshave been exaggerated and/or simplified in places for illustrativepurposes only. One of ordinary skill in the art will appreciate the manypossible applications and variations based on the following examples ofpossible embodiments.

Various embodiments describe a control system for an entertainmentvenue, which provides for the capability of an operator to controldevices such as lighting, video and audio within an entertainmentenvironment in response to environmental control inputs such as a videoimage captured from the observed environment, audio from the observedenvironment, or other control devices or sensors within the observedenvironment such as an Inertial Measurement Unit [IMU], Radio FrequencyIdentification [RFID], temperature, or pressure sensor. In particularembodiments, the control system can track objects within the environmentand trigger events based on motion or position of the object in order tocontrol a device such as a stage light, a video board, an audio outputor any other output device within the environment.

FIG. 1 illustrates an embodiment of a system block diagram for a controlsystem. The control system 100 includes a processor/controller subsystem102, an environment control input subsystem 104, an environment devicecontrol subsystem 106, one or more controlled devices 108, a userinterface 110, a database 112, and a memory 114. In the embodimentillustrated in FIG. 1, the environment control input 104 and theenvironment device control 106 are each in communication with theprocessor/controller 102. The environment control input 104 receives oneor more inputs related to the environment, such as an entertainmentvenue, in which it is located. In response to receiving an environmentcontrol input, the processor/controller 102 processes the environmentcontrol input and provides a control signal to the environment devicecontrol subsystem 106. In response to receiving the control output, theenvironment device control 106 controls the operation of one or morecontrolled devices 108 within the environment 116. In a particularembodiment, the controlled devices 108 may be one or more lightfixtures. The controlling of the controlled devices 108 may includecontrolling the pan and tilt of the lighting device as well ascontrolling the intensity of the output thereof. The user interface 110is in communication with the processor/controller 102 and allows a useror operator of the control system 100 to program behavior of controlleddevices 108 relative to their capability and in response to environmentcontrol inputs 104 as well as perform other functions of the system aswill be further described herein. The database 112 is in communicationwith the process/controller 102 and functions to store data related tovarious aspects of the control system 100. The memory 114 is coupled tothe processor/controller 102 and functions to allow storage of datarelated to the processing of signals as well as comprising a computerreadable medium for storing software to perform the various functions ofthe control system 100 described herein.

In at least one embodiment, the environment includes an objectenvironment having an object space associated therewith. Theprocessor/controller 102 recognizes the presence of a predeterminedobject in the object space. In at least one embodiment, theprocessor/controller 102 receives one or more images of the object spacefrom an imaging device. In a particular embodiment, the one or moreimaging devices are cameras configured to capture one or more images ofat least a portion of the object space. In various embodiments, thepredetermined object is previously selected by a user. Theprocessor/controller 102 then determines a coordinate location of therecognized predetermined object in the object space. In at least oneembodiment, the determining of the coordinate location of thepredetermined object is based upon processing the one or more capturedimages of the object space. In still other embodiments, the determiningof the coordinate location of the predetermined object may be performedby GPS, triangulation, geolocation, location or pressure sensors mountedin or on a floor surface, altitude sensors, IMU sensors, or any otherlocation method. Some embodiments utilize a combination of methods anddevices. In a particular embodiment, the coordinate location is atwo-dimensional coordinate location within the object space. In stillother embodiments, the coordinate location is a three-dimensionalcoordinate location within the object space.

In at least one embodiment, the processor/controller receives adefinition of a target set of coordinates in the object space. In atleast one embodiment, the definition of a target set of coordinates inthe object space is performed by a user in a prior setup procedure aswill be further described herein. The target set of coordinatesrepresents a location within the object space in which it is desired totrigger one or more outputs by the controlled output devices 108 whenpredefined criteria are satisfied. In at least one embodiment, thedefined criteria are related to a spatial relationship, such apredefined proximity, between the coordinate location of thepredetermined object and the target set of coordinates. Theprocessor/controller 102 then determines the spatial relationshipbetween the recognized predetermined object and the target set ofcoordinates and compares the spatial relationship with predeterminedspatial relationship criteria. Examples of predetermined spatialrelation criteria include proximity of the recognized predeterminedobject to the target set of coordinates where the target coordinates mayrepresent another observed or predetermined object, an area or zone, aspecific location, or a line defined as a delimiter or “trip line”within the environment. If the determined spatial relationship criteriafalls within the predetermined spatial relationship criteria, theprocessor sends control signals to modify the object space environment.Examples of predetermined object attribute criteria include location,color, pattern or size of the recognized predetermined object within theobject space. If the predetermined object's attribute criteria fallswithin the predetermined attribute criteria, the processor sends controlsignals to modify the object space environment. Examples ofpredetermined motion criteria include the motion of the predeterminedobject relative to a predetermined path in the environment and ordetecting a gesture of the recognized predetermined object. In variousembodiments, the object space environment is modified by controlling oneor more controlled devices 108 within the environment. In a particularexample, object space environment is modified by controlling the pan,tilt, and light intensity of one or more light fixtures to direct theirlight beams within the object space environment. In other embodiments,the control of devices 108 within the environment may be mapped to audioattributes obtained from the environment such as pitch or volume ormapped to predefined set of environment control attributes defined by asystem profile.

In various embodiments, the system supports various fixture controlprotocols such as the following industry protocols: DigitalMultiplexing—DMX 512=E1.11, USITT DMX512-A (maintained by EntertainmentServices and Technology Association [ESTA]); Remote DeviceManagement—RDM; and Architecture for Controls Networks—ACN=ANSI E1.17(maintained by ESTA).

For fixture control signal data management, leverages several mechanismsto manage/control the new attributes that one can now associate tofixtures. Some of the attribute fields that are utilized in buildingfixture lighting cues today include: dimming (intensity), pan, tilt,color, gobo, iris (beam angle), frost (beam edge control), focus, andshutter/strobe. The Vision-2-Vision system provides newattributes—examples include: attributes that manage object tracking,object selection, V2V-TRAX behavior and rate of change, audio frequencyselect, V2V-SYNC behavior and rate of change, V2V-IMAGE effect(s), andV2V-IMAGE behavior and rate of change. To the user, the new fields canbe thought of as additional fixture channels added to and utilized by anassociated device. In various embodiments, the user interface presents acontrol interface that is consistent with systems today presenting newattributes as additional fixture channels so that users can manage thenew capability in a familiar way. In addition, in some embodiments thesystem offers simplified interfaces to offer a simpler control interfacestrategy for cue management and creation. An example would be to utilizea behavior latch interface tool to link a new feature and controlcapability to mark/pre-set cue in a standard industry cue stack. Anexample of using some of the fields described is shown in the tablebelow (note, example shown in non-tracking cue mode):

FIXTURE DIM- OBJECT OBJECT V2V-TRAX CUE ID MER PAN TILT COLOR GOBOTRACKING SELECT BEHAV./RATE ID MFG- 50  30  25 GREEN SQUARE OFF 1A_UNIT11 MFG- 50 *60 *25 BLUE CIRCLE OBJ_ID12 AUTO- FOLLOWSPOT 1A_UNIT12 3SS MFG- 50  90  25 **GREEN SQUARE OBJ_ID12 MAN-RP COMPLEMENT 1A_UNIT13 *Given V2V-TRAX is in use for the cue, the pan and tilt valuesrepresent a mark/preset cue value. **The console will evaluate theprimary color stored for OBJ_ID12 in the image DB and override the COLORattribute control to select a complementary color.

In some embodiments, solutions that leverage an existing controller thatis used in conjunction with a rack mount/standalone Vision-2-Visioncontroller may leverage one of several strategies to manage fixturecontrol operations. One can use a multiplexed control signal (e.g. DMXuniverse) and connect it to the V2V controller, define/map key fixtureattribute data into the external device controller, or leverage anoverlay approach where devices are programmed via an industry standardprotocol directly to the external device controller.

FIG. 2 illustrates an embodiment of an entertainment environment 200 foruse with the control system 100. The entertainment environment 200includes a stage 202 and a truss structure 204 attached thereto. Thetruss structure 204 functions to suspend various equipment such aslighting fixtures and other devices above the stage 202. A downstageleft camera 206 a, a downstage right camera 206 b, an upstage leftcamera 206 c and an upstage right camera 206 d is attached to the trussstructure 204 and are coupled to the processor 102 as environmentcontrol inputs 104. The cameras 206 a-d are directed downward toward thestage area and capture video and/or still images of the environment 200.One or more stage lights 208 or other devices are affixed to the trussstructure and function as controlled devices 108 under control of theprocessor/controller 102 as further described herein. Although theembodiment illustrated in FIG. 2 uses four cameras, it should understoodthat in other embodiments any number of cameras or imaging devices maybe used. In particular, in one embodiment, the entertainment environmentmay include one imaging device. In some embodiments, other environmentalsensors may be attached to the truss 204 structure such as IMUs,pressure sensors, or lasers, and objects which may appear in the volumeof space defined by the truss 204 and stage 202 may have sensorsattached such as RFID.

FIG. 3 illustrates an embodiment of general software components of thesoftware architecture for the VISION-2-VISION control system. Thegeneral software components include a system configuration managementcomponent 302, a system database management component 304, a systemevent management component 306, a system resource management component308, a system background monitoring component 310, a system debug andmaintenance management component 312, and a user interface managementcomponent 314. In various embodiments, the control system is designed tobe scalable so that the system can grow as the user's needs grow orsystem feature performance demands increase. In a particular embodiment,components can be mapped to a specific CPU core or deployed in anindependent computing platform blade/interface depending on theuser/performance needs. If a particular service or component is hostedon a different platform blade, Ethernet, RS-232, or any other suitableprotocol may be used to communicate data to the master control computinginterface.

FIG. 4 illustrates an embodiment of a general overview of softwarelogical planes for the control system. FIG. 4 illustrates varioussoftware components of the vision-2-vision control system for aparticular embodiment, but it should be understood that it does notrepresent a comprehensive view of such. In other embodiments, additionalor different software components may be present. The software logicalplanes include a platform logical plane 402, a message logical plane404, a management logical plane 406 and an application logical plane408. The platform logical plane 402 may include node or device interfacemanagement functions including initialization, configuration, taskmanagement, diagnostics, fault management, clock management, eventmanagement, resource management, profile management and rend-statemanagement. The message logical plane 404 includes message transportservices. The management logical plane 406 includes system levelmanagement functions such as a initialization, configuration, eventreporting, resource management, fault management, alarm management,profile management, clock management, upgrade and state management. Theapplication logical plane 408 includes device control service functionssuch as object detection and tracking, image processing, devicenormalization, video processing, voice processing and music processing.

FIGS. 5-5′ illustrate an embodiment of logical elements and sequencesthat represent key control services capability of the control system100. It should be understood that FIGS. 5-5′ does not represent acomprehensive view of the software components defined absolute groupingof capability to service, nor sequences managed in the vision-2-visioncontrol system in all embodiments. In other embodiments, differentservices, groupings or sequences may be used. In the embodimentillustrated in FIGS. 5-5′, the control system 100 includes a number ofcore services including V2V-VOX core services 502, V2V-TRAX coreservices 504, V2V-IMAGE core services 506, V2V-SYNC core services 508,and V2V-NOTE core services 510. The V2V-VOX core services 502 providesuch services as voice command, and control grammar. The V2V-TRAX coreservices 504 includes services such as region of interest and objectmanagement. The object management may include identification, size,shape, movement, gesture, and special and orientation attributes. TheV2V-IMAGE core services 506 may include an effect engine, and imagecontrol such as color, shape, texture, brightness mapping andmanagement. The V2V-IMAGE core services 506 may further include videomapping and management. The V2V-SYNC core services 508 may include audioprocessing, genre management, synchronization services, and an effectsengine. The V2V-NOTE core services 510 may include profile management,event logging, environment data and control archiving.

In various embodiments, the voice control capability design provided byV2V-VOX core services 502 is built around infrastructure that preventsthe need for training the system for each user. In some embodiments,multiple language support is possible. The command grammar is based upona set of fields that map to specific terms/tokens. The term may be anattribute of a fixture or desired operations against the activefixture(s). For example, if one says: “lights on”, “turn the lights on”,or simply “on”, one or more fixtures will be turned on. The fixturesthat will come on are based upon a number of factors. If no fixtures hadever been turned on, then all fixtures will come on. If a prior commandhad been issued that turned on a specific set of fixtures, then noaction will occur as the prior command had already turned on fixtures.

In various embodiments, the system tracking control capability providedby V2V-TRAX core services 504 uses cameras for system computer visioncontrol and leverages edge detection and motion detection algorithms tolocate and identify objects as well as obtain object characteristics andattributes. Examples of edge detection algorithms that may be used invarious embodiments are listed in the following table:

Vision Algorithm or Technique Associated Function or Task Boosting,Random Trees, Expectation Machine Learning Maximization, K-nearestNeighbor Canny, Subtraction, Sobel Edge Detection Delaunay Triangulation3D Shape Rendering Delaunay Triangulation Tracking Objects DistanceTransform Image Processing of Edge Detection Frame Differencing,Codebook, Background Subtraction: Averaging Isolating Image BackgroundMethod Objects Gaussian, Laplacian of Gaussian Edge Detection FilterGradient processing Corner Detection Haar Classifier Machine Learning;Face Detection Harris Corner Finding: Extracting Geometric MeasurementsHistogram Equalization Image Processing for Better Contrast HoughTransform Circle Detection & Detection of other Simple Forms HoughTransform Line Detection Inpainting Image Repair Line Fitting 3D PointAnalysis Lucas-Kanade, Horn-Schunck Optical Flow: Tracking & MotionMorphology, Flood Fill, Threshold, Image Segmentation PyramidReprojection Depth Map Generation; 3D Structure of Environment SIFT KeyFeature Detection SLAM Simultaneous Localization and Depth MappingStructure From Motion Camera Trajectory; 3D Structure of EnvironmentSubpixel Corners Corner Finding: Extracting features Time of Flight,Projected Light Camera Type for Depth Watershed Algorithm, Pyramid ImageSegmentation without a Mean-Shift, Pyramid Background Image Segmentation

In a particular embodiment, a detection algorithm based upon a Harristype of edge detection is used. Images are cross-referenced viain-memory database searches that utilize search methods selected toreturn/locate data as efficiently and quickly as possible. In someembodiments, data/image matching is performed based upon evaluating thetracked object attributes against object images in a system database.System control parameters determine how many attributes, of an objectobserved by a camera, must match an object in the database for aconsecutive set of frames to determine whether an object is one thatshould be tracked. Motion detection capability is leveraged tofacilitate automatic selection of an object based upon its location andthe period for which the object resides at a given location. For objectsthat are actively tracked, the system will return the object ID, objectgroup ID, Cartesian coordinates, as well as the observed height andwidth of the object. Some attributes of a given objective can be deriveddirectly from the systems image database based upon the object ID (e.g.color attributes, object complexity factor, etc). The data is used toupdate the value of the attributes sent in the fixture control signal todefine the associated operation. In various embodiments, the systemV2V-TRAX control is protocol agnostic so that any industry protocol canbe supported at the hardware and/or device control layer (e.g. DMX, RDM,ACN). In still other embodiments, the system can support interactionwith devices tracked via RFID methods.

The system will define the event (or object) space via severalmechanisms. The data can be manually entered into the system databasebased upon data from the fixture plot and data associated with thedimensions from the stage that will be used for the event. Alternately,the system can obtain localization data by reference to a number ofpoints obtained during load in or as part of the system init sequenceduring the product setup. Localization data is used to provide themaster set of reference dimensions and data to be used for a givenevent. The data is stored with an event location name/ID for referenceshould a future event be done at the same location.

In various embodiments, image management capability provided byV2V-IMAGE core services 506 is based upon two key areas. One area is thevideo effect engine that applies effects to a provided video stream.Most systems to date utilize canned streams created prior to the eventwhich complicates the ability for the lighting designer to mesh thevideo with the fixtures in real time. Embodiments of the new engineallow the user to create a cohesive look that ties fixture attributes toany projected video. The second area leverages computer visionprinciples associated with image database analysis. This subsystem isused to perform attribute analysis of an object so that fixtureattributes can be defined based upon an observed object's attributes(e.g. use colors in the fixtures that compliment the color of a piece ofmaterial that will be used in a costume). Image mapping controlparameters allow the artist to set how a fixture should map to theattributes of an object and what behavior is desired (e.g. complimentthe attributes, use consistent attributes, use contrasting attributes,etc). In other embodiments, the image management capability is utilizedto process and compare shapes or patterns control devices can create.This facilitates the ability to compare two different devices,evaluating all control capability and attributes of the device, todetermine if one device would be a suitable replacement for the other.

In various embodiments, the audio synchronization subsystem provided bythe V2V-SYNC core services 508 is based upon an audio processing enginethat will split an audio stream out into specific data attributes.Analysis may be done via hardware or software utilizing Fast FourierTransforms [FFTs] for spectrum analysis or any other suitable technique.In some embodiments, the data is accessible via software access methodsdefined on an industry standard protocol (e.g. XML). Various attributescan be collected and stored in audio attribute tokens—examples include:beat, pitch/frequency, key, time, volume/loudness, and harmony.Attributes may be cross linked to data that tracks/records changesacross specific sampling periods—this allows the system to detect ifthere is a change in a given attribute. The audio attributes will thenbe available to the designer to map specific fixture behavior to a givensound characteristic. Sample applications include applications in whichthe designer configures a group of fixtures so that cues trigger basedupon the audio sensitivity within a specified frequency, or the designermay associate how quickly a fixture pans or how bright the light isbased upon the volume of sound. In various embodiments, the audio enginewill be used to trigger operations against video panels to control whichpanels are turned on/off. The display panel control interface will bebased upon a user defined panel matrix that maps the installed locationof the panels against their associated control address—for a set ofpanels used to construct a video wall, the panel matrix could looksomething like a checker board.

In various embodiments, the journaling subsystem provided by theV2V-NOTE core services 510 is based upon detailed table structuredefinitions and relationships coupled with efficient search algorithmsand tools. In some embodiments, templates are used, based upon anindustry standard software language (e.g. XML), for fixture definitionand music semantic data storage. In some embodiments, the templates maybe offered to fixture vendors for defining library plug-ins of theirfixture set. The V2V-NOTE subsystem supports the capability of storingvideo streams observed during a rehearsal or live event. In suchembodiments, the system may require additional storage devices toprevent burdening the control system with the overhead of the associatedstorage. Notes taken during rehearsals/live events can be coupled to alive video stream. In various embodiments, a table structures is definedand utilized to generate numerous system, event/show, configuration,status, and equipment reports. Some reports may be based upon across-section of system events that were posted/reported for a specifiedperiod of time.

The example environmental control input 104 may include: voice, console,keyboard, touch screen, or other physical interface control signals,audio, video, time code or clock (SMPTE, MIDI), signals from a inertialmotion unit (IMU), a signal from an RFID, a pressure sensor or atemperature transducer. The example environmental control input 104 mayfurther include image or object data such as a gesture, shape, size,color, position, and spatial relation.

Examples of environment device control 106 attributes may include:camera attributes such as focus, pan, tilt, zoom, zone event, fixtureattributes such as color, pattern, pan, tilt, test behavior, or trackingmode, video attributes such as layer, pixel map, and pattern, discreteevent attribute triggers such as playing a sound effect enabling snoweffects, etc., a profile containing audio semantics, a thermostat usedto manage temperature in the environment, and or security controlattributes such as camera pan, tilt, zoom, or zone alarms.

The environment control inputs 104 are provided to the various coreservices, which are processed by the core services and used to monitorand update environment device control attributes within the controlenvironment 116. The control system may further generate data 512 thatis stored in the database 112, which may include user or control modeenvironment profiles, device or image profiles, control or genreprofiles, event reports, genre profiles, controlled grammar, objectmodels, environment attributes, device attributes, image attributes,control attributes, audio attributes, event attributes, and system logs.

FIG. 6 illustrates a particular embodiment of software components 600 ofthe V2V-TRAX core services 504. In various embodiments, a complex objectmay have one or more models associated with it. Each model may containspecific details of an object. For an object to be observed by thesystem, a user defined number of details is observed for a user definedset of consecutive frames. Simple objects will have fewer details/modelsas the object will be defined more so by its edges than specific imagedetails of the object. In particular embodiment, a laser can be used toset fixture focus. Additionally, the color of the laser as well as theshape of the image projected by the laser can be used to define/setfixture attributes. An input device to the system in this particularembodiment is a camera, which will see the laser beam and associatedshape. When a camera is present, some embodiments may utilize theobserved shape of a fixture's projected beam of light to manage systeminitialization operations as well as attribute management to variousenvironment control devices. Other embodiments leverage the observedobjects width and height to manage what attributes to update in thevarious control devices within the environment. Given the control system100, comprehends volumes of space, the system will have a 3D effectsengine which defines default control attributes values that generatespecific 3D looks/effects—effects can be dynamic and change in relationto predetermined objects within the space. For example, the system mayprovide a collection of 3-D effects based upon these object's attributessuch as a wave, teardrop, heartbeat and many more.

FIG. 7 illustrates a particular embodiment of software components of theV2V-IMAGE core services 506. In particular embodiments, a complex objectmay have one or more models associated with it. Each model containsspecific details of an object for an object to be observed by thesystem, a user defined number of details may be observed for a userdefined set of consecutive frames. Simple objects will have fewerdetails/models as the object will be defined more so by its edges thanspecific image details of the object. Image processing services mayprovide facilities to search object models for matching characteristicsacross multiple objects. This allows the system to normalize goboselection of cross fixtures by finding similar images. A gobo is aphysical template which is slotted inside or placed in front of alighting source and is used to control the shape of emitted light. Agobo may have a particular color or shape, such as a star, a spiral, aflower, etc., or other shapes and colors as are known in the art.

FIG. 8 illustrates a particular embodiment of software components 800for the V2V-VOX core services 502. The V2V-VOX software componentsprovide voice control of the control system 100. In a particularembodiment, V2V-VOX services have a setting to control which fixtures aparticular voice command will apply to. The interface processes commandstrings constructed from a set of terms that are used to form commandgrammar for the system. The user is able to reorder the sequence andwhich terms are used to alter the command grammar. For example, thisallows the user to determine/configure the system to either accept thecommand “turn on the Martin Macs” or “turn the Martin Macs on” by simplychanging the order of the action and device terms used in the commandgrammar. In the particular embodiment described herein, Martin Macs area particular type of lighting fixture. Other embodiments of the V2V-VOXsoftware provide methods where devices can be categorized into devicetypes (e.g. Martin Mac is device type lighting or profile lighting),this allows the system to apply actions on different devices based uponthe device type. An example of grammar terms and values includes:

ACTION On, off, dim, intensity, pan, tilt, focus, strobe, frost DETAILUp, down, percentage, upstage, downstage, stage right, stage left,reference point 1, reference point 2 and laser DEVICE Fixture name, namefor group of fixtures, name for a group of fixtures based upon location,and name of a group of fixtures based upon fixture type.

In various embodiments, the V2V-VOX core services 502 may support anumber of different languages including English, Canadian French,Spanish, German, Arabic and any other dialect. Supported languages maybe further broken down such that English may be broken down into U.S.English, U.K. English, Australia English, and New Zealand English.Spanish may be broken down into Mexican Spanish, South American Spanish,and European Spanish.

FIG. 9 illustrates a particular embodiment of software components 900for the V2V-SYNC core services 508. The components 900 allow musicgenres to be associated to fixture attribute profiles such as those usedin prior events or from a genre system default database to quicklycreate different looks by setting up the system to utilize a music genreto define fixture behavior and attribute usage. For example, aparticular genre of music may be used to control a particular lightpattern, color, movement, etc. In a particular embodiment, a projectionsystem or video panel may be used such that it can appear to sync tomusic attributes by utilizing the audio processing services to controlwhen the output to specific panels will be enabled.

FIG. 10 illustrates an embodiment of an example of a 4×4 LED panelmatrix under control of the V2V-SYNC software components 900. In thefirst example illustrated in FIG. 10, the panels of the 4×4 LED matrixwill be enabled in columns from left to right with the panels beingturned on in sequence based upon the sync trigger parameters. In thisexample, each tick mark added to the letter “A” represents a consecutivedisplay step (e.g., A″ would follow A′). In the second example, thepanels are enabled in columns from right to left. Panels will be turnedon in sequence based upon the sync trigger parameters. In the thirdexample, the panels will be enabled from left to right by selecting oddnumbered panels. Panels will be turned on in sequence based upon thesync trigger parameters. In the fourth example, panels will be enabledfrom left to right selecting even numbered panels. Panels will be turnedon in sequence based upon the sync trigger parameters.

FIG. 11 illustrates an embodiment of a user interface of the controlsystem 100. The user interface includes a touch screen 1102, a physicalinterface 1104, a voice interface 1106, a laser pointer interface 1108,and a camera interface 1110. The touch screen 1102 allows an operator tointerface with the control system 100 such as designating objects fortracking, designating a region of interest, controlling a fixture shownin the touch screen display or any other functions of the control system100. The physical interface 1104 may include a keyboard, one or morebuttons, sliders, control panels or any other physical interface whichmay be used to control various aspects of the control system 100. Thevoice interface 1106 may include a microphone or headset used to issuevoice commands to the control system 100. Different embodiments of thesystem will utilize different types and combinations of cameras such aslocal control cameras or environmental control cameras. The localcontrol camera 1110 is used not to capture images of the environmentitself but instead used as a local input control interface for theoperator. For example, the operator may make gestures at the localcontrol camera to control the various aspects of the control system 100.In another example the operator may hold up a card printed with an imageof a specific: color, shape, colored shape, or any other object whichwill then be recognized by the camera and used to control variousaspects of the control system 100. The laser pointer 1108 may be used topoint to a particular location within the controlled environment. Forexample, the laser pointer 1108 may be used to point to a specificlocation upon the stage and the control system 100 may recognize thefocus of the laser pointer using the cameras which capture images of theenvironment and designate the particular area pointed to as a region ofinterest to be used in further control of devices such as lightingfixtures. Although various embodiments have been described as using alaser pointer 1108, it should be understood that other embodiments mayuse any type of light source such as a fixture, laser, or laser pointer.

FIGS. 12 a-12 b illustrate a particular embodiment of an operator usingthe laser pointer 1108 to designate a particular region of interest.Referring to FIG. 12 a, the operator points a laser to a location on thestage of the entertainment environment and in response the area definedby the laser pointer as an area or region of interest will be identifiedon the user interface. In the particular example illustrated in FIG. 12a, the area is identified on the user interface by an oval shape. Inanother example, the operator may issue a voice command of “downstagecenter.” In response the control system will identify the downstagecenter portion of the stage as an region of interest on the userinterface using an oval shape. Referring now to FIG. 12 b, in anotherexample of a selection or identification of an object or region ofinterest, the user selects an object displayed on the touch screen 1102and the selected object will be identified using image processing andwill be designated as a selected object in the user interface on thetouch screen 1102. In the particular embodiment illustrated in FIG. 12b, the selected object, which in this case is a person, is identified asbeing selected by the object being surrounded by a rectangle with across sectioned with an X. In still another example, an operator maytouch or select a particular area displayed on the touch screen. Thearea selected will now be identified on the touch screen 1102 of theuser interface as a region of interest. In the particular embodimentillustrated in FIG. 12 b, the entire upper stage is designated as aregion of interest by the selection of a portion of the upper stage bythe operator. In other embodiments, a particular laser image shape orcolor can be mapped to control other attributes such that differentlaser pointers and/or laser pointer colors may be used to controldifferent aspects of the control system 100. In still other embodiments,the operator may select a portion of the environment displayed on thetouch screen 1102 by drawing a border around the desired region ofinterest using the touch screen interface or circling the object ofinterest in the control environment via a laser pointer. The mechanismsand associated attributes used to provide feedback to the user regardinga defined location, region, or object of interest may vary. The use ofan oval to represent a location, a rectangle cross sectioned with an Xfor an object, or filling an area with lines are shown as possibleexamples.

FIGS. 13 a-13 a″ and 13 b-13 b″ illustrate an embodiment of a sequenceof an operator using a focus behavior function of the V2V-TRAX softwarecomponent. The focus behavior function allows the control system 100 toupdate the attributes of devices and the control environment based uponthe position of an object, the color or shape of an object, or both theposition and color or shape of an object. For example, the functionallows an operator to set a pan and tilt value of a particular light orgroup of lights such that the light is positioned to focus at thelocation where an object is observed. Further detail of the V2V-TRAXFocus Behavior function is described in the following table:

Control System Service V2V-TRAX Tool Service Operation Focus BehaviorExample Update the attributes of devices in the control environmentbased upon Control  The position of an object or both the position andcolor or shape of an object Capability  (e.g. set a lights pan and tiltvalues such that the light is positioned to focus at Introduced  thelocation where an object is observed) Key *Control System: Providescomputational power, storage, and user interfaces Components Camera(s):Provides video and image input to computer vision pipeline tasks  May bepackaged with IMU, pressure, laser , and or GPS Laser Pointer: Used topoint to a location in the environment where tool operation is mapped;Used to point to or select (e.g. circle) an object in the environmentExample Additional Data Storage Units: Used to archive video and oraudio streams Additional Head Set: Used to tell the system a location inthe environment (e.g. Don's office, Components Downstage Center) wheretool operation is mapped Utilized Based Laser Pointer: Used to point toa location in the environment where tool operation is Upon Desiredmapped; Used to point to or select (e.g. circle) an object in theenvironment System Touch Panel: Used as a control interface to selectobjects, zones, locations, and lines Services and Sensors: IMU,pressure, RFID, color, and temperature sensors may be used for Controladditional attributes of devices in the environment Capability ExampleLighting Fixtures, LED Panels, Cameras Devices Controlled by the SystemExample Map the tool service, Focus Behavior, to a control device orgroup of devices (e.g. General Tool Lighting Fixture(s)) and to one ormore of the given control device's attributes (e.g. Service LightFixture(s) pan, tilt, and iris size) Provisioning Steps Example ToolBehavior Type: Test, Automatic, Manual Control Activation Type:Automatic, Manual, Validate, Test Parameters Activation Threshold:Control parameter used to control how long a device's parameters areupdated Sequence Control: Control parameter used to manage how thesystem moves between control devices contained in a group (e.g.automatically, manually, by type, by location, for system) ControlThreshold: Control parameter used to manage timing associated with whenthe system moves to a new device in a group Service Tool Type & ControlExample Service Tool Type Device Update Criteria Provisioning MethodsExample Tool Test: After position data is determined to Focus can beconfigured via a system Control focus a device to a location, sequencecontrol interface; the target focus position Behavior through testingother attributes used on can be manually entered, specified via a Typesthe device (e.g. color) voice command, or set using a laser Automatic:Once a device(s) is/are pointer mapped to focus behavior, automaticallyturn on the device, set the focus position, and based upon other controlparameters, move to another device Manual: Once a device(s) is/aremapped to focus behavior, automatically turn on the device, and wait forthe operator to accept or modify then accept the data

FIGS. 13 a-13 a″ and 13 b-13 b″ illustrate an embodiment of a sequencefor setting the focus position using a laser pointer. The sequence isillustrated as using frame IDs to indicate the progression of time. Itshould be understood that a particular frame ID may not refer just toone frame but may, in fact, include multiple frames of a video sequence.In frame ID 1, a baseline image of the entertainment environment iscaptured in a video frame and processed to provide an image of theenvironment to the user interface. In frame ID 2, the user shines alaser pointer in the environment to a desired target focus position. Itshould be noted that the laser beam is shown in the figure for clarity,however, in actual use, only the beam point may be visible. The controlsystem detects the position of the laser pointer and sets a locationobject at that point. In the user interface, the operator is providedwith an indication that the system has selected the particular location.

In the particular embodiment illustrated in FIGS. 13 a-13 a″, theposition is indicated by an oval. In frame ID 3, the location is mappedto a control device or group of devices such as a lighting fixture orfixtures and to one or more of the given control device's attributessuch as light fixture, pan, tilt and iris size. In the user interface, aparticular light fixture is shown as enabled for that location. In aparticular embodiment, the control system may generate an internal logincluding an indication that the criteria have been met and the deviseattribute updates have been initiated. The device control attributes ofthe control system are updated to indicate that mapped environment lightfocus data has been updated. Continuing to FIGS. 13 b-13 b″, additionalframes continue to illustrate the scenario for a sequence where thepositioning laser is present and focus type activation is validated. Inframe ID 4, the system enables a positioning laser and location data forthe positioning laser is obtained. The control system provides avalidation of the target focus position to the user through the userinterface. In frame ID 5, the location is compared to a prior value. Inframe ID 6, the position data is validated by the user interfacedisplaying an indication that the location is mapped to the particularlight fixture. As before, an internal log may be generated indicatingthat the criteria has been met and that devices attribute updates havebeen initiated. In the prior step of the position data being validated,if the focus position has changed from the prior value, the positiondata may be updated with the revised position values. The environmentdevice control attributes are updated to reflect the new mappedenvironment light focus data. FIGS. 13 c-13 c″ illustrate anotherexample sequence for focus behavior.

The embodiment illustrated in FIGS. 13 c-13 c″ show an example sequencefor automatically setting fixture focus utilizing a system positioninglaser. In frame ID 1, the system enables the positioning laser andlocation data for the positioning laser is obtained. The detected objectis then shown as a target focus position to the user. In frame ID 2 thesystem turns on the first lighting fixture in the device control group.The control system then focuses the first lighting fixture at thepositioning laser location that indicates to the user that position datahas been obtained and updates the map environment light focus andrelative position data for the first lighting fixture. In frame ID 3 thesystem turns off the lighting fixture and prepares to move to the nextunit.

FIGS. 13 d-13 d″ illustrate an embodiment of a flow diagram for focusbehavior. Focus behavior can be enabled for a single control device or adevice group. Example focus behavior control parameters include behaviortype, activation type, activation threshold, sequence control andcontrol threshold. Example focus behavior types include test mode,automatic mode or manual mode. In step 1301 a camera input is providedto the control system. In step 1302 it is determined if focused behavioris mapped to environmental control devices. If not, a log event isgenerated in step 1304 and the procedure ends. If yes, the procedurecontinues to step 1306 in which it is determined whether a camera orvideo stream is active. If not, a TRAX event alarm is posted providingdevice information in step 1307 and the procedure ends. If yes, thevideo frame image is captured and processed in step 1308. Example objectattributes that may be obtained in step 1308 include size, color, edges,coordinates or shape of an object. In step 1310, it is determined if anobject is observed in the video image frame. If yes, the image frame isevaluated against prior frames and feedback to this effect is providedas output in step 1312. In step 1316 the control system 100 mayoptionally receive environment monitoring control device input. Thisinput may include data output from an RFID, monitored temperature, amonitored volume level, or any other parameter of the environment whichmay wish to be monitored. In step 1318, if enabled, the environmentaldata is merged with the data received from the evaluation of the imageframe in step 1312. If in step 1310 it is determined that there is noobject observed in the video image frame, the procedure continues tostep 1311 in which it is determined if a target focus position has beenprovided. If no, a log event is generated and the procedure ends. Ifyes, the procedure proceeds to the aforedescribed step 1318. In step1320 the control device, such as a lighting fixture is turned on, theiris setting is set to small, and pan and tilt values of the device areset to match the position of the observed object or specified location.In step 1322 it is determined if the system has a position laser and iffocus activation type is validated. If no, the focus behavior activethresholds are set or updated. In particular embodiments, the focusbehavior active thresholds include a setting of a predetermined timeperiod for which mapped control device attributes are to be updated. Ifin step 1322 it is determined that the system does have a position laserand that focus activation type is validated, the procedure continues tostep 1326 in which the position laser is turned on and the current lightfocus position is audited. The system then returns to step 1324. In step1328 the system and node environment variables are checked. An exampleof such a variables are inhibit and pause. In step 1330 theenvironmental control device attribute data is updated. Thisenvironmental control device attribute data may include light pan andtilt. In step 1332 a particular environment control device iscontrolled. An example of such devices include a camera or a lightingfixture.

FIGS. 14 a-14 a″ illustrate an embodiment of a sequence for a gesturebehavior function of the V2V-TRAX software module. Using such afunction, an operator may update the attributes of devices in thecontrol environment based upon the gesture, color, size, and/or shape ofan object. For example, the control system 100 may be configured tostrobe a particular light if an object gesture is a clap. In still otherembodiments, the gesture, color and/or shape of an object at a givenlocation may be used to update the attributes of devices in the controlenvironment. For example, the control system may be configured to changethe color of a light if an object gesture is a jump and the object islocated in the area of center stage. In still other embodiments, thecontrol system may be configured to trigger events in the environmentbased upon a gesture, color and/or shape of an object. For example, thesystem may be configured to change the color of a light in use if theobject observed is a circle. Other examples of devices which may becontrolled by the system include lighting fixtures, LED panels, cameras,security systems, and thermostats. The gesture behavior function may beused to map a particular gesture to a control device or group of devicessuch as lighting fixtures and to one or more of the given controldevices attributes such as a light fixture's color and iris size.Examples of gestures include jumping, clapping, pointing, sitting,standing or laying down. An individual gesture may be a gesture observedrelative to an individual object, an object in a given area, or as agroup gesture where a gesture observed is relative to a group of objectsor objects in a given area. Further detail of the V2V-TRAX GestureBehavior function is described in the following table:

Control System Service V2V-TRAX Tool Service Operation Gesture BehaviorExample Update the attributes of devices in the control environmentbased upon Control  The gesture, color, and or shape of an object (e.g.strobe a light if an object Capability  gesture is a clap) Introduced The gesture, color, and or shape of an object at a given location (e.g.change  color of light if an object gesture is a jump and the object islocated in the  area of center stage) Trigger events in the environmentbased upon  The gesture, color, and or shape of an object (e.g. changethe color of light in  use if the object observed is a circle) Key*Control System: Provides computational power, storage, and userinterfaces Components Camera(s): Provides video and image input tocomputer vision pipeline tasks  May be packaged with IMU, pressure,laser , and or GPS Example Additional Data Storage Units: Used toarchive video and or audio streams Additional Head Set: Used to tell thesystem a location in the environment (e.g. Don's office, ComponentsDownstage Center) where tool operation is mapped Utilized Based LaserPointer: Used to point to a location in the environment where tool UponDesired operation is mapped; Used to point to or select (e.g. circle) anobject in the System environment Services and Touch Panel: Used as acontrol interface to select objects, zones, locations, and Control linesCapability Sensors: IMU, pressure, RFID, color, and temperature sensorsmay be used for additional attributes of devices in the environmentExample Lighting Fixtures, LED Panels, Cameras, Security Systems,Thermostats Devices Controlled by the System Example Map the toolservice, Gesture Behavior, to a control device or group of devices (e.g.General Tool Lighting Fixture(s)) and to one or more of the givencontrol device's attributes (e.g. Service Light Fixture(s) color andiris size) Provisioning Steps Example Tool Behavior Type: Individual,Group Control Activation Type: Automatic, Manual, Copy, Auto-Specific,Rotation/Angle Parameters Activation Threshold: Control parameter usedto qualify when attribute updates are qualified for posting/sendingClear Threshold: Control parameter used to qualify when attributeupdates should stop Duration Type: Control parameter used to specifycomplex or linked operations Duration Threshold: Control parameter usedto manage duration of updates or control based upon duration typeGesture: Control parameter used to specify the gesture type for anobject (examples: jump, clap, point, sit, stand, lay down) Service ToolType & Control Example Service Tool Type Device Update CriteriaProvisioning Methods Example Tool Individual: Gesture observed relativeto an Gesture recognition can be configured Control individual object orobject in a given area via a system control interface Behavior Group:Gesture observed relative to a Types group of objects or objects in agiven area

Referring to FIGS. 14 a-14 a″, in a frame ID 1, the system has mappedthe object gesture pointing to represent a person holding their arm outin a given direction. In frame ID 2, an object is detected and anindication of the detected object is shown in the user interface. Inframe ID 3, the observed object holds his or her arm out towards theaudience and points in a particular direction. In response the systemobtains a gesture pointing object attribute. The control system thenupdates the device associated with that gesture by positioning thelighting fixture in the direction pointed to by the observed object.This is performed by mapping the environment attribute gesture to sendpan, tilt and focus data to the selected control lighting fixture.

FIGS. 14 b-14 b′ illustrate a flow chart of an embodiment of the gesturecontrol behavior. In step 1402, the control system 100 receives a camerainput which includes one or more images of the control environment. Instep 1404 it is determined whether there is gesture behavior mapped toenvironmental control devices. Gesture behavior can be enabled for asingle control device or a device group. Examples of gesture behaviorcontrol parameters include behavior type, activation type, activationthreshold, clear threshold, duration type, duration threshold andgesture. Example gesture behavior types may include individual or groupbehavior types. If the answer is determined to be no in step 1404, a logevent is generated and the procedure ends. If it is determined to be yesin step 1404, the procedure continues to step 1406 in which it isdetermined whether the camera video stream is active. If the cameravideo stream is not active, a TRAX event alarm is posted related deviceinformation is generated and the procedure ends. If the camera videostream is indicated as active, the procedure continues to step 1408 inwhich the video frame image is captured and processed to determinewhether there are any objects observed in the frame. Example objectattributes which may be obtained during the processing include size,color, type, edges, coordinates, area/zone and shape.

In step 1410 it is determined if there is an object observed in thevideo frame image. If the answer is no then in step 1412 it isdetermined whether there is an existing gesture event active. If not, alog is generated and the procedure ends. If yes, the procedure continuesto step 1414. If in step 1410 it is determined that an object isobserved in the video image frame, the procedure also continues to step1414 and a TRAX event is posted indicating that an object has been foundmay be generated and output to the user. In step 1416 optionalenvironment control device data may be put into the system and in step1418, if enabled, the environmental monitoring device data may be mergedwith the information obtained from processing the video frame image. Anexample of environmental monitoring device data may include data from anIMU. In step 1420 it is determined whether criteria for a certaingesture has been met. If not, the process continues to step 1422 inwhich gesture behavior clear thresholds are either set or updated and alog event is generated and a procedure ends. If it is determined in step1422 that gesture criteria have been met, the system continues to step1424 in which gesture behavior active thresholds are either set orupdated. In step 1426 system and node environment variables are checked.Examples of variables include inhibit and pause. In step 1428environmental control device attribute data is updated. Examples ofattribute data include light pan and tilt parameters. In step 1430,environment control devices are controlled in accordance with the deviceattribute data. Example devices include cameras, lighting fixtures andvideo streams.

FIGS. 15 a-15 d″ illustrate an embodiment of sequences of a proximitybehavior function of the V2V-TRAX software module. The proximitybehavior function allows the update of attributes of devices in thecontrol environment based upon the relationship, i.e., distance, betweenobjects observed in the environment. For example, light behavior likecolor or intensity can be controlled based upon the distance betweenobjects. The proximity behavior function can also update attributes ofcontrol devices based upon the distance between an object and areference point or area or based upon the size, color, shape, motionand/or direction of an object observed in the environment. For example,selecting a particular light color based upon the height or color of anobserved object. In other embodiments the attributes of devices in thecontrol environment can be updated based on parameters that define aspace or operational containment area. For example, a dark zone couldrepresent an area where a television camera may reside so light beamsshould not take a direct path across the area to prevent camera spotblindness.

The attributes of devices in the control environment may also be updatedbased upon attributes defined for devices in a specified zone. Forexample, based upon attributes defined for devices in a specifiedreference zone, the attributes for devices specified in the target zonemay be mirrored, copied, inverted or other changes may be applied. Forexample, attributes can be updated to use the same colors present in thespecified reference zone in other environment target zones by performinga copy operation. The proximity behavior function may also be used totrigger events in the environment based upon the location of an objectobserved in the environment. For example, a trigger event may be set toturn on lights when an object reaches a specific location or to changeLED panel colors based upon the location of an object relative to agiven panel or to a set of panels. A trigger event may be set based upona relationship, i.e., distance, between objects observed in theenvironment. For example, the system may be configured to turn offparticular lights when selected objects are more than five feet apart.The distance relationship can be relative to two objects or between anobject and a reference point or area. Other trigger events may be setbased upon the size, color, shape, motion and/or direction of an objectobserved in the environment. For example, the system may be configuredto dim lights when motion of a particular object is detected upstage. Instill other embodiments, a trigger event may be set based upon anobjects relationship to a specific line. For example, the system may beconfigured to start a snow effect when an actor walks past a doorthreshold. Further description of the V2V-TRAX Proximity Behaviorfunction is described in the following table:

Control System Service V2V-TRAX Tool Service Operation ProximityBehavior Example Update the attributes of devices in the controlenvironment based upon Control  The relationship (i.e. distance) betweenobjects observed in the environment Capability  (e.g. light behavior,like color or intensity, can be controlled based upon the Introduced distance between objects; relationship can also be based upon thedistance  between an object and a reference point or area)  The size,color, shape, motion, and or direction of an object observed in the environment (e.g. light color can be selected based upon the height orcolor of an  object observed)  The parameters that define a space oroperational containment area (e.g. a dark  zone can represent the areawhere a television camera may reside so light beams  should not take adirect path across the area to prevent camera spot blindness)  Theattributes defined for devices in a specified zone   Mirror, copy,invert, or apply other changes to devices in the control   environmentbased upon the attributes defined for the specified zone (e.g.   use thesame colors present in the specified zone in other environment   zonesby performing a copy operation) Trigger events in the environment basedupon  The location of an object observed in the environment (e.g. turnon lights when  an object reaches a specific location, change LED panelcolors based upon the  location of an object relative to a given panelor set of panels)  The relationship (i.e. distance) between objectsobserved in the environment  (e.g. turn off lights when objects morethan 5′ apart); the distance relationship  can be relative to twoobjects or between an object and a reference point or area  The size,color, shape, motion, and or direction of an object observed in the environment (e.g. dim lights when motion of object is upstage)  Objectrelationship to a specific line (e.g. start snow when actor walks pastdoor  threshold) Key *Control System: Provides computational power,storage, and user interfaces Components Camera(s): Provides video andimage input to computer vision pipeline tasks  May be packaged with IMU,pressure, laser, and or GPS Example Additional Data Storage Units: Usedto archive video and or audio streams Additional Head Set: Used to tellthe system a location in the environment (e.g. Don's office, ComponentsDownstage Center) where tool operation is mapped Utilized Laser Pointer:Used to point to a location in the environment where tool operation isBased Upon mapped; Used to point to or select (e.g. circle) an object inthe environment Desired Touch Panel: Used as a control interface toselect objects, zones, locations, and lines System Sensors: IMU,pressure, GPS, RFID, color, and temperature sensors may be used forServices and additional attributes of objects in the environment ControlCapability Example Lighting Fixtures, LED Panels, Cameras, SecuritySystems, Thermostats Devices Controlled by the System Example Map thetool service, Proximity Behavior, to a control device or group ofdevices (e.g. General Tool Lighting Fixture(s)) and to one or more ofthe given control device's attributes (e.g. Light Service Fixture(s)Focus Parameters = Pan and Tilt) Provisioning Steps Example ToolBehavior Type: Zone, Location, Trip Line, Object Control ActivationType: Automatic, Manual, Copy, Auto-Specific, Rotation/Angle ParametersActivation Threshold: Control parameter used to qualify when attributeupdates are qualified for posting/sending Clear Threshold: Controlparameter used to qualify when attribute updates should stop DurationType: Control parameter used to specify complex or linked operationsDuration Threshold: Control parameter used to manage duration of updatesor control based upon duration type Control Curve: Control parameterused to manage operation across a group of devices to normalize orsmooth the capability (e.g. pan or tilt control curve value smoothing)Service Tool Type & Control Example Service Tool Type Device UpdateCriteria Provisioning Methods Example Tool Zone: An object has entered adefined zone A zone can be a predefined region of Control or area in thecontrol environment and the space, defined by outlining the areaBehavior control constraints have been met (e.g. the with a laserpointer, by telling the Types object stayed within the zone the requiredsystem the name of a zone, or by amount of time) specifying an area viaa control interface Location: An object is at a defined location in Alocation can be a predefined spot, the control environment and thecontrol defined by pointing a laser pointer to a constraints have beenmet (e.g. the object location, by telling the system a specific stayedat the location for required time) location, or by specifying thelocation via a control interface Trip Line: An object has crossed a linethat A trip line can be a predefined line delimits or defines a givenlocation in the location, drawn with a laser, specified environment bytelling the system a line location, or specifying the location via acontrol interface Object: An object is at a defined distance An objectcan be predefined to the from another object, zone, location, or tripsystem, selected by the user telling the line and the controlconstraints have been system which object to use, or by met selecting anobject via a control interface

FIGS. 15 a-15 a″ illustrate an embodiment of an example sequence showinga light group operation mapped to an observed objects predefined targetlocation. In frame ID 1, a baseline image of the entertainmentenvironment is displayed. In frame ID 2, an object is detected withinthe environment and the size, location, color, shape or edges of theobject are acquired as object attributes. In the user interface, anindication is provided that an object has been detected. In frame ID 3as the object moves across the stage, object attributes includingtrajectory, motion, angle and location are obtained. When the objectmoves to a predefined location, a light group mapped to that specificlocation is turned on. In this example, the predefined target locationfor the object is represented by an oval.

FIGS. 15 b-15 b″ illustrate an embodiment of a sequence for updating LEDpanels based upon the location of an observed object relative to apredefined area or zone. In some embodiments, the location of anobserved object could be determined from pressure sensors assuming thefloor structure of the stage was equipped with them. In the embodimentillustrated in FIGS. 15 b-15 b″, LED panels are placed on the stage.When a detected object moves into an enabled target area or zone, one ormore LED panels are enabled causing them to light up. In this example,the predefined target location for the object is represented by an oval.

FIGS. 15 c-15 c′ illustrate an embodiment of a sequence for managing howattributes are set on a mapped lighting fixture based upon theattributes of a fixture in a preselected reference zone or area. Inframe ID 1 a user turns on a light and sets the color, location andshape of the light. In the particular illustration, the light is locatedin a Zone 1 and is shaped as a star. The star is detected as an objectand attributes of color, shape and location are obtained. An indicationis given to the user that the object has been detected in Zone 1. Inframe ID 2, the user sets a zone mapping mode and sets a function tomirror Zone 1 to Zone 2. The system then mirrors the settings from Zone1 to Zone 2. In frame ID 3, the user sets the zone mapping mode to copyZone 1 to Zone 2 and a user is given an indication that the object hasbeen detected and copied from Zone 1 to Zone 2.

FIGS. 15 d-15 d″ illustrate an embodiment of a sequence showing how atrip line location may be utilized to trigger an alarm indicator if anobject moves past the defined trip line. In frame ID 1 a baseline imageof an office environment is shown. In frame ID 2 the user defines a tripline location using the user interface and maps the trip line locationto a device alarm attribute. An object is detected in frame ID2—attributes of the object include location size, shape, and location.When an objected detected within the environment moves to a locationpast the trip line based on object attributes such as trajectory, motionand location, a trip line alarm is enabled.

FIGS. 15 e-15 e′ illustrate an embodiment of a flow chart for theproximity behavior function of the V2V-TRAX software module. In step1502 an input, such as from a camera, audio, laser, image or keyboard,is obtained by the control system 100. In step 1504 it is determinedwhether a proximity behavior has been mapped to environmental controldevices. Proximity behavior can be enabled for a single control deviceor a device group. Example proximity behavior control parameters includebehavior type, activation type, activation threshold, clear threshold,duration type, duration threshold and curve. Example proximity behaviortypes may include zone, location, trip line or object. If in step 1504it is determined that the answer is no, a log event is generated and theprocedure ends. If the answer in step 1504 is yes, the process continuesto step 1506 in which it is determined whether the camera video streamis active. If no, a TRAX alarm event is posted and the procedure ends.If yes, the procedure continues to step 1508 in which a video frameimage is captured and processed. During processing the attributes of anyobjects within the video frame image are obtained. These objectattributes may include size, color, type, edges, coordinates, area/zoneor shape. In step 1510 it is determined whether a new object is observedin the video frame image. If not, the procedure continues to step 1512in which it is determined whether an existing proximity event is active.If no, a log event is generated and the procedure ends. If yes, theprocedure continues to step 1514. If the answer in step 1510 is yes,then the procedure also continues to step 1514. In step 1514 the imageframe is evaluated against prior frames to determine additional objectattributes such as trajectory and motion. In an optional step, 1516,environment monitoring control device data may be input. If enabled, instep 1518, this environmental monitoring device data may be merged withthe object attributes previously obtained. Additional object data thatmay be obtained by the environment monitoring control device includesangle, orientation, velocity and temperature of an object. In step 1520it is determined whether proximity criteria has been met. If not, theprocedure continues to step 1522 in which proximity behavior clearthresholds are set or updated. A log event is then generated and theprocedure ends. If in step 1520 the proximity criteria has been met, theprocedure continues to step 1524 in which proximity behavior activethresholds are set or updated. In step 1526 system and node environmentvariables are checked. Examples of environment variables include inhibitand pause. In step 1528 the environment control device attribute datasuch as light pan and tilt is updated. In step 1530 environment controldevices are controlled in accordance with the proximity data obtained bythe procedure. Examples of control devices include cameras, lightingfixtures, video, thermostats and other discrete signals such as alarmsand the activation of pyrotechnic displays.

FIGS. 16 a-16 b″ illustrate an embodiment of a tracking behaviorfunction for the V2V-TRAX software module. Using the tracking behaviorfunction, the attributes of the devices in the control environment maybe updated based upon the location of an object observed in theenvironment. For example, moving lights will track an object similar tothe operation of follow spots. In other embodiments, the attributes ofdevices may be updated based upon the relationship, i.e., distance,between objects observed in the environment. For example, light trackingcan be conditionally controlled based upon the distance between objects.Relationships can also be based upon the distance from the object and areference point or area. The attributes of devices in the controlenvironment can be updated based upon the size, color, shape, motionand/or direction of an object observed in the environment. For example,a tracking light color can be selected based upon the height or color ofan object observed. The tracking behavior function may be also used toset trigger events advancing environment attributes based upon thelocation of an object observed in the environment. For example, a systemcan be configured to begin tracking and turn on lights when an objectreaches a specified location or change LED panel colors based upon thelocation of an object relative to a given panel or several panels.

In other embodiments, trigger events in an environment may be based uponthe relationship, i.e., distance, between objects observed in theenvironment. For example, the system may be configured to turn offlights, i.e., stop tracking when objects are more than five feet apart.The distance relationship can be relative to two objects or between anobject and a reference point or area. In still other embodiments,trigger events in the environment may be based upon the size, color,shape, motion and/or direction of an object observed in the environment.For example, a system may be configured to dim lights when motion of anobject is upstage. Further description of the V2V-TRAX Tracking Behaviorfunction is described in the following table:

Control System Service V2V-TRAX Tool Service Operation Tracking BehaviorExample Update the attributes of devices in the control environmentbased upon Control  The location of an object observed in theenvironment (e.g. moving lights will Capability  track an object similarto the operation of follow spots) Introduced  The relationship (i.e.distance) between objects observed in the environment  (e.g. lighttracking can be conditionally controlled based upon the distance between objects; relationship can also be based upon the distancebetween an  object and a reference point or area)  The size, color,shape, motion, and or direction of an object observed in the environment (e.g. the tracking light color can be selected based uponthe height  or color of an object observed) Trigger events in theenvironment based upon  The location of an object observed in theenvironment (e.g. begin tracking and  turn on lights when an objectreaches a specific location, change LED panel  colors based upon thelocation of an object relative to a given panel or set of  panels)  Therelationship (e.g. distance) between objects observed in the environment (e.g. turn off lights (i.e. stop tracking) when objects more than 5′apart); the  distance relationship can be relative to two objects orbetween an object and a  reference point or area  The size, color,shape, motion, and or direction of an object observed in the environment (e.g. dim lights when motion of object is upstage) Key*Control System: Provides computational power, storage, and userinterfaces Components Camera(s): Provides video and image input tocomputer vision pipeline tasks  May be packaged with IMU, pressure,laser , and or GPS Example Additional Data Storage Units: Used toarchive video and or audio streams Additional Head Set: Used to tell thesystem a location in the environment (e.g. Don's office, ComponentsDownstage Center) where tool operation is mapped Utilized Based LaserPointer: Used to point to a location in the environment where tooloperation is Upon Desired mapped; Used to point to or select (e.g.circle) an object in the environment System Touch Panel: Used as acontrol interface to select objects, zones, locations, and linesServices and Sensors: IMU, pressure, RFID, color, and temperaturesensors may be used for Control additional attributes of devices in theenvironment Capability Example Lighting Fixtures, LED Panels, Cameras,Security Systems, Thermostats Devices Controlled by the System ExampleMap the tool service, Tracking Behavior, to a control device or group ofdevices (e.g. General Tool Lighting Fixture(s)) and to one or more ofthe given control device's attributes (e.g. Light Service Fixture(s)Focus Parameters = Pan and Tilt) Provisioning Steps Example ToolBehavior Type: Continuous, Contingent Control Activation Type:Automatic, Manual, Copy, Auto-Specific, Rotation/Angle ParametersActivation Threshold: Control parameter used to qualify when attributeupdates are qualified for posting/sending Clear Threshold: Controlparameter used to qualify when attribute updates should stop DurationType: Control parameter used to specify complex or linked operationsDuration Threshold: Control parameter used to manage duration of updatesor control based upon duration type Control Curve: Control parameterused to manage operation across a group of devices to normalize orsmooth the capability (e.g. pan or tilt control curve value smoothing)Service Tool Type & Control Example Service Tool Type Device UpdateCriteria Provisioning Methods Example Tool Continuous: Tracking, oncetriggered, will Continuous tracking can be configured Control operatecontinuously via a system control interface Behavior Contingent: Thetrigger to track an observed Contingent tracking can be configured Typesobject is contingent upon a control via a system control interfaceparameter (e.g. distance to another object, or location in environment)

FIGS. 16 a-16 a″ illustrates an embodiment of a sequence showing lightbeams tracking the position of an object. In frame ID 1 a baseline imageis obtained. In frame ID 2 an object is detected and a group update isenabled to set the device attributes to map a light group to the object.Environment device control attribute updates are sent to update the pan,tilt and focus data on each of the lights within the light group tofocus their respective lights on the detected object. As illustrated inframe ID 3, as the object moves across the stage, the lights continue totrack it.

FIGS. 16 b-16 b″ illustrates an embodiment of a sequence in which a userselects moving scenery as an object to track by circling it with thelaser pointer. If the moving scenery is comprised of one piece withclear boundaries the laser can simply be pointed to the piece ofscenery. In frame ID 1 the user places scenery in the environment. Inthe particularly illustrated embodiment, the scenery is a car. In frameID 2 the user circles the car scenery with a laser pointer. In responsethe control system 100 detects the object and maps light focus data suchas pan and tilt to the detected object. As can be seen in frame ID 3, asthe object moves through the environment the light focus is updated tofollow the object based upon trajectory and motion attributes obtainedfrom the object.

FIGS. 16 c-16 c′ illustrates an embodiment of a procedure for thetracking function behavior of the V2V-TRAX software module. In step1602, the control system 100 receives an input for example, from acamera, audio input, laser, image or keyboard. In step 1604 it isdetermined whether tracking behavior is mapped to environmental controldevices. The tracking behavior can be enabled for a single controldevice or a device group. Example behavior control parameters includebehavior type, activation type, activation threshold, clear threshold,duration type, duration threshold and curve. Example tracking behaviortypes include continuous and contingent. If the answer in step 1604 isno, a log event is generated and the procedure ends. If the answer instep 1604 is yes, the procedure continues to step 1606 in which it isdetermined whether the camera video stream is active. If not, a TRAXalarm event is generated and the procedure ends. If yes, the procedurecontinues to step 1608. In step 1608 the video frame image is capturedin process to obtain object attribute data of any object detected withinthe video frame image. The object attributes may include, for example,size, color, type, edges, coordinates, area/zone or shape. In step 1610it is determined if a new object is observed in the video image frame.If no new object is observed in the video image frame, the procedurecontinues to step 1612 where it is determined if an existing trackingevent is active. If not, a log event is generated and the procedureends. If an existing tracking event is active, the procedure continuesto step 1614 in which the image frame is evaluated against prior imageframes to obtain object attribute data such as trajectory and motion. Ifthe answer to step 1610 is yes, the procedure still continues to step1614. In step 1616 a TRAX event is posted, indicating the detection of anew object, is provided to the user. In step 1618 environment monitoringcontrol device inputs may optionally be obtained. In step 1620, if it isenabled and there is environment monitoring control device data present,it is merged with the previously obtained object attribute data.Examples of environment monitoring control device data may include datafrom an IMU, RFID or temperature. This additional object data obtainedfrom the environment monitoring control devices may include angle,orientation, velocity and temperature. In step 1622 it is determinedwhether the tracking criteria has been met. If not, the procedurecontinues to step 1624 which sets or updates the tracking behavior clearthresholds. A log event is generated, and the procedure ends. Iftracking criteria is met in step 1622, the procedure continues to step1626 in which tracking behavior active thresholds are set or updated. Instep 1628 system and node environment variables are checked. Examplesvariables may include an inhibit or pause variable. In step 1630 controldevice attribute data, such as light pan and tilt, is updated. In step1632 environment control devices are controlled in accordance with thecontrol device attribute data to correspond with any behavior trackedwithin the environment.

FIGS. 17 a-17 a″, 17 b-17 b′, 17 c-17 c″, 17 d-17 d″, 17 e-17 e″, and 17f-17 f illustrate embodiments of a compare behavior function of aV2V-IMAGE software component. The compare behavior function allows forupdate of the attributes of devices in the control environment basedupon the color of an image defined or shown to the system by an operatoror user. For example, the color of a light beam can be set to match,compliment or contrast the color of an image shown to the system. Thecompare behavior function further allows updates of the attributes ofdevices in the control environment based upon the shape or pattern of animage defined or shown to the system. For example, the shape or colormay be mapped to an LED panel, a video stream layer or be used to setthe GOBO pattern of a light based upon comparison to the image shown tothe system. The compare behavior function also allows the comparison ofdevice attributes identifying devices with similar or matchingcapability. When device inventory changes, it can be used to determinewhich devices are best suited to replace the original equipmentinventory. For example, by comparing colors, GOBO patterns, pan and tiltcapability and other attributes. Further description of the V2V-IMAGECompare Behavior function is provided in the following table:

Control System Service V2V-IMAGE Tool Service Operation Compare BehaviorExample Update the attributes of devices in the control environmentbased upon Control  The color of an image defined or shown to the system(e.g. set the color of a light Capability  beam to match, compliment, orcontrast the color of an image shown to the Introduced  system)  Theshape or pattern of an image defined or shown to the system (e.g. mapthe  shape or color to an LED panel, a video stream layer, or to set thegobo pattern of  a light based upon comparison to the image shown to thesystem) Compare device attributes to find devices with similar ormatching capability  When inventory changes, determine which devices arebest suited to replace the  original equipment inventory (e.g. comparecolors, gobo patterns, pan and tilt  capability, and other attributes)Key *Control System: Provides computational power, storage, and userinterfaces Components Camera(s): Provides video and image input tocomputer vision pipeline tasks  May be packaged with IMU, pressure,laser, and or GPS Example Additional Data Storage Units: Used to archivevideo and or audio streams Additional Head Set: Used to tell the systema location in the environment (e.g. Don's office, Components DownstageCenter) where tool operation is mapped Utilized Based Laser Pointer:Used to point to a location in the environment where tool operation isUpon Desired mapped; Used to point to or select (e.g. circle) an objectin the environment System Touch Panel: Used as a control interface toselect objects, zones, locations, and lines Services and Sensors: IMU,pressure, GPS, RFID, color, and temperature sensors may be used forControl additional attributes of objects in the environment CapabilityExample Lighting Fixtures, LED Panels, Cameras, Video Streams DevicesControlled by the System Example Map the tool service, Compare Behavior,to a control device or group of devices General Tool Obtain environmentcontrol device capability profiles or descriptions Service Obtain imagesof reference control shapes, colors, or patterns Provisioning StepsExample Tool Behavior Type: Color, Shape, Pattern, Combination ControlMax Match Threshold: Control parameter used to define the maximum targetmatch Parameters comparison value required Min Match Threshold: Controlparameter used to define the minimum target match comparison valuerequired Operation Mode: Set, Normalize, Match Match Time Threshold:Control parameter used to manage the duration for comparing objects BestMatch: Parameter used to maintain an index of the top matches ServiceTool Example Service Tool Type Behavior Type Provisioning MethodsExample Tool Color: The color of the baseline object is Compare behaviortype can be Control compared to the supported color pallet in aconfigured via a system control interface Behavior device or is used toset the mapped device to Types a similar color, complimentary color, orcontrasting color Shape: The shape of the baseline object is compared tothe supported shapes in a device or is used to set the mapped device toa similar shape Pattern: The pattern of the baseline object is comparedto the supported patterns in a device or is used to set the mappeddevice to a similar pattern Combination: The type is used whennormalizing or looking for similar devices - indicates a suite ofattributes will be compared

FIGS. 17 a-17 a″ illustrate an embodiment of a sequence for setting alight GOBO pattern and color on a mapped device based upon an objectshown to the system by a user. In frame ID 1, the user shows a card witha star on it to a camera or alternately configures the system to use animage of the card as a baseline reference image. The image of the cardis processed to determine color and shape object attributes. In frame ID2, the system has successfully detected the object and compared it to anobject stored in the database which is mapped to a particular light GOBOpattern. In frame ID 3, the light GOBO pattern associated with thedetected object is enabled and environment device control attributes areupdated to map the environmental light shape and color set.

FIGS. 17 b-17 b′ illustrates an embodiment of a sequence for definingthe baseline image without requiring a user to touch the user interface.In frame ID 1, the user shows a card with a star on it to the camera. Inframe ID 2, the user begins to rotate the card and the system detectsobject attributes including color, shape and texture. In addition, theuser is notified that the object has been detected. In frame ID 3 theuser continues to rotate the card and the object attributes of color,shape and texture are obtained therefrom. In response to the rotation ofthe card by the user, the image on the card is configured as a baseline.

FIGS. 17 c-17 c″ illustrate an embodiment of a sequence for setting alight GOBO pattern and color on a mapped device based upon the objectobserved by the system by an enabled light fixture. In frame ID 1, thesystem sets a pattern and color on a light fixture and turns it on. Thecontrol system then obtains the color and shape of the pattern from thecaptured video image and indicates to the user that the pattern has beendetected as an object. In frame ID 2, the object has been detected andsuccessfully compared to a matching object within a database. In frameID 3, the control system turns off the light fixture and enables thedevice having a light GOBO pattern and color matching that of the firstlight fixture.

FIGS. 17 d-17 d″ illustrate an embodiment of a sequence for setting alight GOBO pattern, color and beam position on a mapped device basedupon the object observed by the system via an enabled light fixture.This example sequence is similar to that of FIGS. 17 c-17 c″ except thatthe system, in addition to obtaining object attributes of color andshape, also obtains an object attribute of location of the pattern ofthe first light fixture. Upon successful object detection and comparisonof the light pattern with a matching light pattern of another lightpattern of another fixture within the database, the matching lightfixture is enabled and the environmental device controller attributesare updated to set the shape, color and position that the light fixtureis directed to be the same as that of the first light fixture.

FIGS. 17 e-17 e″ illustrate an embodiment of a sequence for setting anLED panel video layer pattern and color on a mapped device based upon anobject observed by the system via an enabled light fixture. In frame ID1, the system sets a pattern and color on a light fixture and turns iton. The control system then processes a video image of the environmentto obtain color and shape object attributes of the pattern. In frame ID2, the control system compares the detected object with other patternsin a database. In frame ID 3, the system turns off the light fixture.The system then enables device updating for the LED panel video layerpattern and sets the LED video panel layer shape and color to displayone or more patterns corresponding to the pattern projected by the lightfixture.

FIGS. 17 f-17 f′ illustrate an embodiment of a procedure for the comparebehavior function of the V2V-IMAGE software component. In step 1702, thecontrol system receives an input containing a designation of an imagewhich is desired to be defined as a baseline image. The input mayinclude an input from a camera, an input of a device profile, a storedimage, or a command from a keyboard. In step 1704, a baseline image isdefined based upon the input data. In some embodiments, the comparebehavior can be enabled for a single control device or in otherembodiments for a device group. Example compare behavior controlparameters may include behavior type, maximum match threshold, minimummatch threshold, operation mode, match time threshold or best match.Example compare behavior types may include color, shape, pattern or acombination of these. In step 1706, it is determined whether comparebehavior has been mapped to one or more environmental control devices.If the answer is no, the procedure ends. If the answer is yes, theprocedure continues to step 1708 in which it is determined whether thecamera video stream is active and the current mode is a normalized ormatch mode. If the answer in step 1708 is yes, the process continues tostep 1710 in which the video frame image is captured. Then, in step1712, the baseline image is processed and compared to the new image. If,in step 1708, it is determined that the camera video stream is notactive, the process continues to step 1714 in which it is determinedwhether there are images available for matching and the mode is set tonormalize or match. If the answer to step 1714 is yes, the processcontinues to step 1716 in which an image is selected. From step 1716 theprocess continues to step 1712 as previously described. Example imageattributes that may be processed in step 1712 include color, shape,pattern and size. If it is determined in step 1714 that there are noimages available, the procedure continues to step 1718 in which thebaseline image is processed and attributes are applied to control one ormore devices. The attributes may include, for example, shape, color, panor tilt. From step 1712 the procedure continues to step 1720 in whichcontrol parameters are updated. In step 1722 it is determined whetherthe operation mode is set to normalize device. Normalizing devices canbe used to compare attributes to different devices in a controlenvironment to identify devices with comparable capability. For example,comparative capability may include devices that have comparable colors,patterns, pan and tilt or strobe support. If the answer in step 1722 isno, the procedure continues to step 1718 which has been previouslydescribed above. If the answer to step 1722 is yes, the procedurecontinues to step 1724 in which it is determined whether the currentdevice is the last device to be normalized. If this is the last deviceto be normalized, the procedure returns to the aforementioned step 1718.If this is not the last device to be normalized, the procedure continuesto step 1726 in which a new device can be selected and then to thepreviously described step 1708. After step 1708 is performed, theprocedure continues to step 1728 in which an environment control deviceis controlled. Example devices which may be controlled may includelighting fixtures and video streams.

FIGS. 18 a-18 a′, 18 b-18 b′, 18 c-18 c′, and 18 d illustrateembodiments of command behavior for the V2V-VOX software module. Thecommand behavior function allows the attributes of the devices in thecontrol environment to be updated based upon a valid command spoken tothe system via a microphone or headset. Example commands may include“lights on,” “Don's office,” and “downstage center.” Further descriptionof the V2V-VOX Command Behavior function is provided in the followingtable:

Control System Service V2V-VOX Tool Service Operation Command BehaviorExample Update the attributes of devices in the control environmentbased upon Control  A valid command spoken to the system via amicrophone or headset (e.g. Capability  command “lights on”, “Don'soffice”, “downstage center”) Introduced Key *Control System: Providescomputational power, storage, and user interfaces Components Voice userinterface Example Additional Data Storage Units: Used to archive videoand or audio streams Additional Head Set: Used to tell the system alocation in the environment (e.g. Don's Components office, DownstageCenter) where tool operation is mapped Utilized Based Laser Pointer:Used to point to a location in the environment where tool Upon Desiredoperation is mapped; Used to point to or select (e.g. circle) an objectin the System environment Services and Touch Panel: Used as a controlinterface to select objects, zones, locations, and Control linesCapability Sensors: IMU, pressure, GPS, RFID, color, and temperaturesensors may be used for additional attributes of objects in theenvironment Example Lighting Fixtures, LED Panels, Cameras, VideoStreams Devices Controlled by the System Example Map the tool service,Command Behavior, to a control device or group of devices General ToolService Provisioning Steps Example Tool Language Mode: Control parameterused to set default language mode - Control example options: automatic,English, French, Spanish Parameters Privilege Level: Control parameterused to manage the user privilege level for the capability OperationMode: Control parameter used to manage general command operations -example options: audible feedback, feedback tone, feedback message MatchTime Threshold: Control parameter used to manage the duration forsearching for a valid command grammar Noise Threshold: Control parameterused to set audio filter levels

FIGS. 18 a-18 a′ illustrate an embodiment of a sequence for voiceactivation of devices based upon device manufacturer. Beam position isalso set by voice activation in this example. The user says a voicecommand of “turn on Martin Macs.” It should be noted that Martin Macs isa particular manufacturer of a control device that is a control devicehaving a type of a lighting fixture. The control system 100 thenperforms a grammar check on the spoken command to identify an actiontoken of “on” and a device token of “Martin Macs.” In response, thedefault voice command device is set to the Martin Macs. In addition, theenvironment device control attributes are updated to set the Martin Macsshutter or iris to open. The user next issues a command of “centerstage.” The control system then performs a grammar check on the commandto identify a detail token of “center stage”. The device token is thedefault previously set. In this case, the Martin Macs. In response tothis command, the light pan and tilt values are updated on the MartinMacs to position the beam of the Martin Macs to the center stage. Theuser may then issue a command of “turn lights off” The control system100 performs a grammar check on the command to identify an action tokenof “off” and a device token of the default, the default being the MartinMacs. In response to this command, the Martin Macs shutter or iris areset to close.

FIGS. 18 b-18 b′ illustrate an embodiment of a sequence for voiceactivation based upon device manufacturer in which multiple types ofdevices are commanded. Beam position is also set by voice activation. Inthe particular embodiment illustrated in FIGS. 18 b-18 b′, the lightingfixture control devices include Martin Macs and VL1000s, which areanother type of lighting fixture. The user first issues a command of“turn on Martin Macs.” The control system 100 performs a grammar checkto identify an action token of “on” and a device token of “Martin Macs.”The default voice command device is set to Martin Macs, and theenvironment device control attributes are updated to set the shutter oriris on the Martin Macs to open. The user then issues another voicecommand of “center stage” followed by a voice command of “turn onVL1000s.” In response, the control system 100 performs a grammar checkto identify a detail token first of “center stage” and a device token ofthe default, the default being the Martin Macs. In addition, an actiontoken of “on” and a device token of “VL1000s” is identified. In responseto the center stage command, the environment device control attributesare updated on the Martin Macs to set the light pan and tilt values tocenter stage. In addition, the VL1000s shutter or iris is set to openand the default voice command device is set to the VL1000s. The userthen gives a voice command of “turn lights off.” The control system 100performs a grammar check to identify an action token of “off” and adevice token of default, the default devices now being the VL1000s. Inresponse to the command, the VL1000s shutter or iris is set to close.

FIGS. 18 c-18 c′ illustrate an embodiment of a sequence for voiceactivation of multiple devices based upon device manufacturer as well asthe setting of beam position and a video layer by voice activation. Inthe example illustrated in FIGS. 18 c-18 c′, control devices of MartinMacs and VL1000s lighting fixtures as well as a layer 1 control devicewhich is a video fixture. A user first says a command of “turn on MartinMacs” and the control system 100 performs a grammar check to identify anaction token of “on” and a device token of “Martin Macs.” In responsethe environment device control attributes are updated such that theMartin Macs shutter or iris is set to open. In addition, the defaultvoice command device is set to Martin Macs. In a next time period, theuser says a command of “center stage” followed by a command of “turn onVL1000s,” then followed by another command of “turn on video layer 1.”For the first command the control system 100 performs a grammar check toidentify a detail token of “center stage” and a device token of thedefault, that is the Martin Macs. For the second command an action tokenof “on” is identified followed by a device token of VL1000. For thethird command, an action token of “on,” a device token of “video” and adetail token of “layer 1” is identified. In response to these commands,the light pan and tilt values of the Martin Macs are updated to directthe Martin Macs to the center stage. In response to the turn on VL1000scommand, the VL1000 shutter or iris is set to open and the default voicecommand device is sent to the VL1000s. In response to the command ofturn on video layer 1, the video layer 1 stream is turned on and thedefault voice command video device is set to layer 1. In a next timeperiod, the user says the command “turn lights off.” The control system100 performs a grammar check to identify an action token of off and adevice token of the default, which, in this case, is the VL1000s. Inresponse to the command “turn lights off,” the VL1000 shutter or iris isset to close.

FIG. 18 d illustrates an embodiment of a procedure for the commandbehavior of the V2V-VOX software component. In step 1802 a voice commandis received by the control system 100. The voice command in someembodiments may include, for example, a voice command input into amicrophone, a headset or a command type for the keyboard. In step 1804command grammar is defined. By default, the command behavior is appliedto the current active device or group of devices. Example commandbehavior control parameters include language mode, privilege level,operation mode, match time threshold and noise threshold. In step 1806it is determined whether a voice has been detected. If a voice has notbeen detected the procedure ends. If a voice has been detected, theprocedure continues to step 1808 in which the words of the voice commandare processed against the command grammar defined in the system. In step1810 it is determined whether a valid command has been identified. Ifthe answer is no, the procedure continues to step 1812 in which it isdetermined whether a match threshold has elapsed. If the matchedthreshold has not elapsed, the procedure returns to step 1808 in whichthe words continue to be processed against the command grammar. If it isdetermined that the match threshold has elapsed, the procedure continuesto step 1814 in which it is determined whether the operation mode is setto audible feedback. Referring again to step 1810, if a valid command isidentified, a log message is generated and the procedure continues tothe aforementioned step 1814. In step 1814, if it is determined that theoperation mode is audio feedback, the procedure continues to step 1816in which a feedback tone or confirmation audio is played back to theuser to confirm that a valid command has been identified. The procedurethen continues to step 1818 in which system and node environmentvariables are checked. Examples of variables that may be checked includean inhibit variable and a pause variable. If it is determined in step1814 that the operation mode is not set to audible feedback, theprocedure also continues to step 1818. In step 1820, the environmentcontrol device attribute data of the device identified by the command isupdated and a VOX event is posted. Examples of the environment controldevice attribute data may include an iris setting, pan and tilt. In step1822 the updated environment control device attribute data is used tocontrol one or more environment control devices. Examples of environmentcontrol devices may include lighting fixtures and video stream boards.

FIG. 19 illustrates an embodiment of the audio behavior function of theV2V-SYNC software module. The audio behavior function allows theattributes of devices in the control environment to be updated basedupon an audio profile associated with a specific genre of music, or oneor more discrete audio attributes. For example, the video streams on agiven video layer may be changed based upon the beat of an audio stream.In another example, the light fixture intensity may be set based uponthe volume changes in an audio stream. In step 1902, an audio stream isreceived by the control system 100. In step 1904 the audio stream isselected by the user. Audio behavior can be mapped to a device or groupof devices. In step 1906 the music characteristics of the audio streamare mapped to audio attributes. Example audio attributes may includebeat, pitch or frequency, key, time, volume and harmony.

In step 1908 an indication is output to the user to indicate themapping. In at least one embodiment, the mapping of the musiccharacteristics to audio attributes may be performed by the user. As toother embodiments, the mapping of the musical characteristics to audioattributes may be automatically performed by the system. In step 1910,the audio attributes are mapped to a control device. The mapping of theaudio attributes to the control device may be performed by the user, oralternately automatically performed by the system. Example audiobehavior control parameters may include sync method, device controlmode, operation, attribute change threshold, device change threshold,genre and label. In step 1912, the system and node environment variablesare checked. Example environment variables include inhibit and pause. Instep 1914, the environment control device attribute data is updated forthe particular control devices and a SYNC event is posted. In step 1916,the environment control devices are controlled according to the updatedenvironment control device attribute data. Example environment controldevices may include lighting fixtures and video streams.

Further description of the V2V-SYNC Audio Behavior function is providedin the following table:

Control System Service V2V-SYNC Tool Service Operation Audio BehaviorExample Control Update the attributes of devices in the controlenvironment based upon Capability  The audio profile associated with aspecific genre of music Introduced  Based upon one or more discreteaudio attributes (e.g. change the video  streams on a given layer basedoff of the beat or set the light fixture intensity  based upon thevolume changes) Key Components *Control System: Provides computationalpower, storage, and user interfaces Audio stream or recorded musicExample Additional Data Storage Units: Used to archive video and oraudio streams Additional Head Set: Used to tell the system a location inthe environment (e.g. Don's office, Components Downstage Center) wheretool operation is mapped Utilized Based Laser Pointer: Used to point toa location in the environment where tool operation Upon Desired ismapped; Used to point to or select (e.g. circle) an object in theenvironment System Services Touch Panel: Used as a control interface toselect objects, zones, locations, and and Control lines CapabilitySensors: IMU, pressure, GPS, RFID, color, and temperature sensors may beused for additional attributes of objects in the environment ExampleDevices Lighting Fixtures, LED Panels, Video Streams Controlled by theSystem Example General Map the tool service, Audio Behavior, to acontrol device or group of devices Tool Service Provisioning StepsExample Tool Sync Method: Control parameter used to track the audioattributes mapped to a Control given device Parameters Device ControlMode: Control parameter that defines the type of sync control datamanagement - examples include profile, cross linked, mapped, thresholdOperation Mode: Control parameter used to define the method used totrigger updates to a mapped device - examples include random,sequential, static Attribute Change Threshold: Parameter used withoperational mode to manage when sync operations may move to a new audioattribute value or used as the threshold value to control when totrigger updates on the mapped device (e.g. volume increases 10 decibelsover a 10 second period) Device Change Threshold: Parameter used withoperational mode to manage time interval between control device updatesGenre: Audio category assigned based off of the characteristics of theaudio stream Label: Parameter used to store an alternate genre or userspecified label

FIGS. 20 a-20 a″, 20 b-20 b″, and 20 c-20 c′ illustrate embodiments of a2D_INIT mode behavior function of the V2V-TRAX software module. The2D_INIT mode behavior function allows for updating of the pan and tiltattributes of devices in the control environment based upon pan and tiltvalues initialized in the 2D_INIT mode operations. FIGS. 20 a-20 a″ and20 b-20 b″ illustrate an example sequence of initializing a lightfixture for the environment. In this particular operation, systemcontrol parameters to define area shape are used to set the shape equalto a square or alternately a rectangle and the plane of control is setto the XY plane. In a first frame ID, the user defines four cornerlocations in a baseline image. The four defined corners define an areato which a particular light fixture is to be limited in its travelrelative to the associated plane. During the process, focus points usedto populate device system parameter corner data (i.e. device pan andtilt values) are obtained for each corner of the shape defined in thetarget control plane. In frame ID 2, the user triggers the 2D_INITsequence for a particular light fixture. In response, the system turnson the fixture and the system detects the beam as an object. Inaddition, object attributes are obtained of size, shape and location ofthe beam. In frame ID 3, the system updates the pan and tilt values tomove the fixture beam over the first corner. As a result the systemobtains motion and location object attributes of the beam. Having foundthe first corner location, the system maps the light pan and tilt of thecurrent position of the light fixture for corner 1 into the devicecorner data system control parameters.

In frame ID 4 the system updates the fixture pan and tilt values to movethe fixture beam over to the second corner. The system monitors the beamand obtains object attributes of motion and location and the currentlight pan and tilt values. Once the beam has reached corner 2, the lightfixture pan and tilt values are mapped as corner 2 system controlparameters. Similarly, in frame ID 5, the system updates the fixture panand tilt values to move the fixture beam over the third corner. Thesystem obtains the object attributes of motion and location associatedwith the current position of the beam and the current light pan and tiltvalues. Once the beam has reached the location for corner 3, thedevice's pan and tilt values are stored in the system corner controlparameters. Finally, in frame ID 6, the system updates the fixture panand tilt values to move the fixture beam over to the fourth corner andthe system detects the location of the beam to obtain the objectattributes of pan and tilt, having found the fourth corner. The systemmaps the light pan and tilt values of the current position of the beamto the corner 4 system control parameters. Having stored the valueswhich represent the 4 corners of the defined control space in the XYplane for a given mapped control fixture, the system can utilize thedata to position the associated fixture within the plane.

Further description of the V2V-TRAX 2D_InitMode Behavior function isprovided in the following table:

Control System Service V2V-TRAX Tool Service Operation 2D_InitModeBehavior Example Update the pan and tilt attributes of devices in thecontrol environment based Control upon Capability  The 2D_InitModeoperations pan and tilt values Introduced Key *Control System: Providescomputational power, storage, and user interfaces Components Camera(s):Provides video and image input to computer vision pipeline tasks  May bepackaged with IMU, pressure, laser, and or GPS Laser: Used to define thecorners of the initialization region Example Additional Data StorageUnits: Used to archive video and or audio streams Additional Head Set:Used to tell the system a location in the environment (e.g. Don'soffice, Components Downstage Center) where tool operation is mappedUtilized Based Laser Pointer: Used to point to a location in theenvironment where tool Upon Desired operation is mapped; Used to pointto or select (e.g. circle) an object in the System environment Servicesand Touch Panel: Used as a control interface to select objects, zones,locations, and Control lines Capability Sensors: IMU, pressure, RFID,color, and temperature sensors may be used for additional attributes ofdevices in the environment Example Lighting Fixtures, Cameras DevicesControlled by the System Example Map the tool service, 2D_InitModeBehavior, to a control device or group of devices General Tool (e.g.Lighting Fixture(s)) and to one or more of the given control device'sattributes Service (e.g. Light Fixture(s) Focus Parameters = Pan andTilt) Provisioning Steps Example Tool Behavior Type: Continuous,Contingent Control Activation Type: Automatic, Manual, Single Step,Group Based Parameters Device ID: Control parameter used wheninitialization is based upon a group (i.e. activation type is GroupBased) Device Corner Data: Array of data used to store device pan andtilt values for a given corner location On-Off Mode: Control parameterused to manage how a fixture beam is turned on or off (example values:power, iris, shutter) Area Shape: Control parameter used to capture theshape defined by the corners (e.g. square) Plane: Control parameter usedto capture the plane in which the corners reside (e.g. X-Y) Service ToolType & Control Example Service Tool Type Device Update CriteriaProvisioning Methods Example Tool Continuous: 2D_InitMode, oncetriggered, Parameter can be configured via a Control will operatecontinuously system control interface Behavior Types Contingent: Thetrigger the 2D_InitMode is contingent upon the value of the activationtype

FIGS. 20 c-20 c′ illustrate an embodiment of the 2D_INIT behaviorfunction of the V2V-TRAX software. In step 2002 a camera or laser inputis received by the control system 100. In step 2004 it is determinedwhether 2D_INIT mode behavior is mapped to one or more environmentcontrol devices. If the answer is no, a log event is generated and theprocedure ends. If the answer is yes, the procedure continues to step2006 in which it is determined if the camera video stream is active. Ifthe camera video stream is not active, the procedure ends and a TRAXalarmed event is posted. If the camera video stream is active, theprocedure continues to step 2008. Referring back to step 2004, 2D_INITmode behavior can be enabled for a single control device or a group ofdevices. Example 2D_INIT mode control parameters include behavior type,activation type, device ID, device corner data, on-off mode, area shapeand plain. Examples of 2D_INIT mode behavior types include continuousand contingent. If it is determined in step 2006 that the camera videostream is active, the procedure continues to step 2008 in which thecorners of the areas to initialize are defined. In a particularembodiment, a system validation laser can be used to set the four cornerlocations. In step 2010 the fixture is turned on or the iris associatedwith the fixture is opened. In a particular embodiment, the focus of thefixture may be set to a narrow tight beam for improved accuracy. In step2012 the image frame is evaluated to obtain object attributes such assize, color and coordinates within the XY plain. In step 2014 it isdetermined if the fixture beam is centered over the corner location. Ifthe answer is no in step 2016, the pan and tilt values are updated toposition the beam over center and the procedure continues back to step2012 where the image frame is again evaluated. If in step 2014 it isdetermined that the answer is yes, the procedure continues to step 2018in which it is determined whether this is the last corner to be defined.If this is not the last corner to be defined, the procedure continues tostep 2020 in which the pan and tilt values are updated to move the beamto the next corner location and the procedure returns to step 2012 inwhich the image frame is again evaluated. If in step 2018 it isdetermined that this is the last corner to be defined, the procedurecontinues to step 2022 in which the fixture is turned off or the iris isclosed. In step 2024 it is determined whether this is the last fixturein a group to be initialized. If the answer is no, the procedurecontinues to step 2026 in which the environment control device attributedata for the current fixture is updated. Examples of this control deviceattribute data include light pan and tilt. The procedure then moves tostep 2028 in which the next light fixture in the group is selected andthe procedure returns to step 2010. If it is determined in step 2024that this is the last fixture in the group, the procedure continues tostep 2030 in which an output is provided to the user indicating that thelast fixture has been initialized and the procedure continues to step2032 in which the environment control device attribute data such aslight pan and tilt for the last fixture is updated. The procedure thencontinues to step 2034 in which one or more environment control devicesare controlled.

FIG. 21 illustrates an embodiment of a procedure for a profile behaviorfunction of the V2V-NOTE software component. The profile behaviorfunction allows a user to generate a profile archive that captures thelight cues or attribute cues on a given show. The profile behaviorfunction further allows a user to generate a profile archive thatcaptures the light cues or attributes and video used on a given show.Still other functions of the profile behavior function includes allowinga user to generate a profile archive that captures the light cues orattribute cues on a given show mapped to a specific genre. For example,the light attributes for a show with a music genre of rock. In anotherfunction of a profile behavior function, show or event profiles that arestored can be used as the basis to generate a new show where similaroperations or behavior is desired. In step 2102, stored video, audio,light queues and show data are provided to the control system 100.

In step 2104, a profile generation type is selected. Example profilebehavior control parameters can include behavior type, error management,duration threshold, duration type, user ID and label. Example profilebehavior types include audio, lights, video, show or combined. In step2106 stored data is selected for profile creation. In step 2108 it isdetermined whether selected file streams are closed. If the answer isno, an alarm event is provided to the user and the procedure ends. Ifthe selected file streams are closed, the procedure continues to step2110 in which the user can initiate or continue profile creation. Instep 2112 it is determined if there were errors reported duringgeneration. If the answer is yes, the procedure continues to step 2114in which it is determined whether a continue on error flag is set. Ifthe flag is not set, a log is generated and the procedure ends. If theflag is set, a log event is generated and the procedure continues backto step 2110. If in step 2112 there were no errors reported duringgeneration, the procedure continues to step 2116 in which the profile isstored in a location in a database based upon user ID and label. Afterstep 2116 a NOTE event is posted and the procedure ends.

Further description of the V2V-NOTE Profile Behavior function isprovided in the following table:

Control System Service V2V-NOTE Tool Service Operation Profile BehaviorExample Control Generate a profile archive that captures the light cuesor attributes used on a Capability given show Introduced Generate aprofile archive that captures the light cues or attributes and videoused on a given show Generate a profile archive that captures the lightcues or attributes used on a given show mapped to a specific music genre(e.g. the light attributes used for a show with a music genre of rock)Generated show or event profile archives can be used as the basis togenerate a new show where similar operations or behavior is desired KeyComponents *Control System: Provides computational power, storage, anduser interfaces Archived/saved show data, video streams, and or audioExample Additional Additional Data Storage Units: Used to archive videoand or audio streams Components Utilized Head Set: Used to tell thesystem a location in the environment (e.g. Don's Based Upon Desiredoffice, Downstage Center) where tool operation is mapped System Servicesand Laser Pointer: Used to point to a location in the environment wheretool Control Capability operation is mapped; Used to point to or select(e.g. circle) an object in the environment Touch Panel: Used as acontrol interface to select objects, zones, locations, and linesSensors: IMU, pressure, RFID, color, and temperature sensors may be usedfor additional attributes of devices in the environment Example DevicesControlled by the System Example General Define the type of profilecriteria desired and error management operation Tool ServiceProvisioning Steps Example Tool Behavior Type: Audio, Lights, Video,Show, Combined = Control Parameters types of profiles (e.g. shows) ErrorManagement: Control parameter used to store flags that determine whatoperations or steps should be taken if an error is encountered DurationThreshold: Control parameter used to define the maximum duration allowedto generate a profile Duration Type: Control parameter used to managewhat mechanism is used to control the duration threshold (e.g. user withmanual start stop indications, internal timer) User ID: Controlparameter used to internally generate a profile label based upon systemor node user information Label: Control parameter used to store a customlabel for the profile; the name is specified by the user

Although not illustrated in the figures, a further function that may beperformed by the V2V control system may include a symmetry behaviorfunction. In the symmetry behavior function the attributes and devicesin the control environment may be updated based upon changes in thedesigned look of an image. For example, a particular image may be formedwith symmetric beams and the symmetry behavior function will maintainsymmetry of the image on device failure. In this example, an image iscreated with light which is supposed to be symmetric. One of thefixtures in the group fails. If the device count is even, that is ifthere were an even number of fixtures in the image, the control systemwill evaluate how many fixtures are still operating in the image. Ifmore than two fixtures are still working, the system will close the irisor shutter on the failed unit's mate device so that it also appears off.In this way, symmetry of the image is maintained.

Further description of the V2V Symmetry Behavior function is provided inthe following table:

Control System Service V2V-Standard Tool Service Operation SymmetryBehavior Example Control Update the attributes of devices in the controlenvironment based upon Capability Introduced  Changes in the designedlook of an image (e.g. image with symmetric  beams)  Example Sequence =Maintain Symmetry on Device Failure:   Assume    an image created withlights is supposed to be symmetric    one of the fixtures in the groupfails    the device count is even (i.e. there were an even number offixtures in  the image)   Control System Actions    control system willevaluate how many fixtures are still operating in  the image    if morethan two fixtures are still working, the system will close the  iris orshutter    on the failed unit's mate device so that it also appears offKey Components *Control System: Provides computational power, storage,and user interfaces Camera(s): Provides video and image input tocomputer vision pipeline tasks  May be packaged with IMU, pressure,laser, and or GPS Example Additional Additional Data Storage Units: Usedto archive video and or audio streams Components Utilized Head Set: Usedto tell the system a location in the environment (e.g. Don's Based UponDesired office, Downstage Center) where tool operation is mapped SystemServices and Laser Pointer: Used to point to a location in theenvironment where tool Control Capability operation is mapped; Used topoint to or select (e.g. circle) an object in the environment TouchPanel: Used as a control interface to select objects, zones, locations,and lines Sensors: IMU, pressure, RFID, color, and temperature sensorsmay be used for additional attributes of devices in the environmentExample Devices Light Fixtures, Video Panels Controlled by the SystemExample General Tool When a set of attributes is mapped to a cue (i.e. aspecific look) additional Service Provisioning parameters are utilizedto store the designed image look type (e.g. random, Steps symmetric,asymmetric) Example Tool Control Behavior Type: Automatic, ManualParameters Look Type: Random, Symmetric, Asymmetric, Fan, Custom DeviceCount: Control parameter used to indicate how many fixtures are used inthe image Fixture Gap: Control parameter used to control spacing offixture beams in a given look (e.g. for a look type of fan, space thebeams using a specific angel of separation between each of the beams)Error Management: Control parameter used to store flags that determinewhat operations or steps should be taken if an error is encountered

It will be appreciated by those skilled in the art having the benefit ofthis disclosure that this Vision 2 Vision control system provides asystem and method for controlling output devices based upon thedetection of objects in an entertainment environment. It should beunderstood that the drawings and detailed description herein are to beregarded in an illustrative rather than a restrictive manner, and arenot intended to be limiting to the particular forms and examplesdisclosed. On the contrary, included are any further modifications,changes, rearrangements, substitutions, alternatives, design choices,and embodiments apparent to those of ordinary skill in the art, withoutdeparting from the spirit and scope hereof, as defined by the followingclaims. For example, although certain embodiments are described for usein an entertainment environment, it should be understood that in stillother embodiments, the control system may be used in any type ofenvironment—an example of an alternate control environment could be abuilding where controlling the environment facilitates the integrationof systems building systems such as lighting, HVAC, and security. Thus,it is intended that the following claims be interpreted to embrace allsuch further modifications, changes, rearrangements, substitutions,alternatives, design choices, and embodiments.

What is claimed is:
 1. A method for controlling an object space havingan associated object space environment, comprising the steps of:defining a target set of coordinates within the object space to definean area of interest within the object space by the steps of: directing alight source with a predetermined target designating image to adesignated target location within the object space by a user tosuperimpose the target designating image onto the object space,capturing an image of the object space including the superimposed targetdesignating image, capturing from the image of the object space thesuperimposed target designating image directed to the designated targetlocation by the light source, and determining the target set ofcoordinates of the captured superimposed target designating imagerelative to the captured image of the object space; recognizing thepresence of a predetermined object in the object space by the steps of:capturing an image of the predetermined object, and detecting thepresence of the predetermined object within the object space using thecaptured image of the predetermined object; determining a coordinatelocation of the recognized predetermined object in the object space;determining a spatial relationship between the recognized predeterminedobject and the target set of coordinates; comparing the determinedspatial relationship with predetermined spatial relationship criteria;and if the determined spatial relationship falls within thepredetermined spatial relationship criteria, modifying the object spaceenvironment.
 2. The method of claim 1, wherein the coordinate locationis a two-dimensional coordinate location.
 3. The method of claim 1,wherein the coordinate location is a three-dimensional coordinatelocation.
 4. The method of claim 1, wherein the object space environmentis a visual environment.
 5. The method of claim 1, wherein the objectspace environment is an audio environment.
 6. The method of claim 1,wherein the object space environment is a sensor environment.
 7. Themethod of claim 1, wherein the step of modifying the object spaceenvironment comprising controlling at least one output device within theobject space environment.
 8. The method of claim 7, wherein thecontrolling of the at least one output device comprises controlling atleast one control attribute of the at least one output device.
 9. Themethod of claim 8, wherein the at least one control attribute includesone or more of, a pan value, a tilt value, and an output intensity valueof the at least one output device.
 10. The method of claim 7, whereinthe at least one output device comprises at least one of a lightingfixture, a camera, a video screen, a video stream, an audio outputdevice, and an effects device.
 11. The method of claim 1, wherein thestep of directing a light source with a predetermined target designatingimage comprises directing a beam of a light source to the designatedtarget location by a user.
 12. The method of claim 1, further comprisingstoring data representing at least one of the object space and thetarget set of coordinates.
 13. The method of claim 1, wherein thedefining of the target set of coordinates in the object space furthercomprises: selecting the designated target location by a user using auser interface device; and determining the target set of coordinatesbased upon the selected designated target location.
 14. The method ofclaim 13, wherein the user interface device comprises at least one of atouch screen and a light source.
 15. The method of claim 1, furthercomprising detecting a gesture of the predetermined object within theobject environment.
 16. The method of claim 1, further comprising:determining attributes of the recognized predetermined object; andcomparing the attributes of objects to those of predetermined attributecriteria, and if the determined attribute criteria falls within thepredefined attribute criteria, modifying the object space environment.17. The method of claim 16, wherein the attributes include at least oneof color, size, shape, pattern, texture, trajectory, motion, angle,height, and temperature.
 18. The method of claim 1, further comprising:determining the characteristics of an audio stream and mapping thecharacteristics to audio attributes; comparing audio attributes withpredefined audio mapping criteria; and if the determined audio criteriafalls within the predefined audio mapping criteria, modifying the objectspace environment.
 19. The method of claim 1, further comprising:determining a set of command tokens; comparing the command tokens to apredefined set of tokens criteria; and if the determined command tokenscriteria falls within the predefined set of tokens criteria, modifyingthe object space environment.
 20. The method of claim 1, wherein thetarget set of coordinates includes at least one of a boundary and zonein the object space.